DOCUMEHT HESUME 

ED Vl4 991 • 95 ^ EA 007 736 

t» 

&0THOP McCutchan^ Sancy; And Others \ 

TITLE ^ Source Book of . Training Products in' Instructional 

Planning and Management, Far West Laboratory Series 

in Instructional Planning, 
INSTITUTION Far West Lab. for Educational 'Research and 

Development ^\ San Fra^acisco^ Calif. 
SPONS AGENCY , National Inst\ of Education (DHEW) r Washington^ * 

D.C. 

PUB DATB 75 • ^ 

NOTE *92p, ' ' . 

AVAILABLE FROM Far West Laboratory for Educational Research and^ 

Development^ 1855 Folsom Streets San Francisco^ 

California 94103 ($3,95) \ ^ 

^EDRS PRICE MF^$0,76 HC-$U.43 Plufe Postage ) 

DESCRIPTORS Communication (Thought Transfer) ; Educational 

Administration; "Educational Planning; Elementary 
Secondary Education; Insttuctional Design; 
♦Instructional Materials; *Instructional Programs;^ 
Instructional Staff; Management; Merchandise 
\ Information ; Planning ; Preschool Education ; *Program 
^ Descriptions; Program Development; Program 

Evaluation; *Resource Guides; Resource Materials ; 
♦Staff Improvement; Training; Training Objectives; ' 
Training Techniques 

ABSTRACT' - 

This sourcebook was developed to^ provide school 
personnel with efficient and effective access to information on 
available training products, it should- be viewed as a resource to 
in9rease awareness and understanding of- new pr oducts^-products that 
train personnel in setting program goals^ pJLanning fox development^ 
and'' performing eva^luation. There are three main sections. The first 
section^ Orientation^ presents a view ofs instructional planning and 
management held by the Far West Laboratory, This section also 
includes the criteria by which "the programs apd resources described 
were selected. The second section^ Program Reports^ describes 15 
program packages that can be implemented withip^ school district to 
meet specific needs for staff development or training, A chart 
organizes key information about all 15 programs so that each can be 
reviewed and compared quickly and easily. Each report should provide 
the decision-maker with enough information to decide whether to 
■ preview the program*' The third section^ Training Resources^ present:^ 
an* annotated list of 27 supplementary materials. These rjesources are 
mostl^y ref-erence books^ handbooks^ textbooks^ monographs^ and case - 
studies that are organized topically under five major areas, 
(Author/IRT) 



Documents accjulred by ERIC include many informal unpubliihed materiali not available from other iourcei. ERIC makes every 
effort to obtain the best copy available. Neverthelew, itemr^f marginal reproducibility are often encountered and thli affecti the' 
aualitv of the microfiche and hardcopy reproductions ERIC makes available via the ERIC Document Reproduction Service (EDRS). 
i^n 1^ not responsible for the quality of the original document. Reproductions supplied by EDRS are the best that can be made from 
fcKjLnal. 



ON 





SOURC 



training products 

in 

instructional planning 
and management 



U S OEPAKTMENTOI 
EDUCATION » 
NATIONAL INS 

EDUCtV^ION 

•this DOCUWENT/HAi/ been repro- 
duced EXACTl/ AS/RECEIVED FROW 
THE PERSON OS'ORiAniZATION ORIGIN- 
ATING IT POINTS G(f VIEW OR OPINIONS 
STATED Do yblVNECESSARILy REPRE- 
SENT OFFIC/al^ATIONAL INSTITUTE OF 
EDUCATION POSITION OR POLICY 



PERMISSION TO BEPBODUCE THIS COPV- 
BIGHTED M*TEBIAL HAS BEEN^BANTEO BY 

TO ERIC AND ORGANIZATIONS OPEflATING | 
UNDER AGREEMENTS WITH THE NATIONAL IN- 
STITUTE OF EDUCATION FURTHER REPRO- 
DUCTION OUTSlOE THE ERIC 'SYSTEM RE- 
QUIRES PERMISSION OF THE COPYRIGHT 
OWNER 



Nancy HcCutcl^an 
Ri ta Fernandez 
Jean C'oleman 



Far Qest Laboratory for Educational Research and Development 

1855 Folsom Street 
San Francisco, Cal i forni a 9^103 



This SouPoeBook was created by the Far West Laboratory for Educational Research 
and Development, a public nonprofit organization funded in part by ^the National 
Institute of Education. Th^ opinions expressed here do not necessarily reflect 
the opinion of th6 funding agency nor emdarsement by any other government agency. 

The Laboratory was established through a Joint Powers Agreement in February 1966. 
Signatories, as of January 1975, i nclude The Regents of the Un i vers i ty of Cali- 
fornia; The Cal i fornia 'State Board of Education; The Trustees of the CaHfornia 
University and State Colleges; The Board of Education^ of the San Francesco 
Uhifi'ed School Di s tr i ct ; , The Regents of the University of Nevada; The Nevada- 
State Board of Education; The Board of Regents, of the Uni vers i ty of Utah; and 
The Utah State Board of Education. 

The, Laboratory's mission is to carry out surveys^vresecirch, development, and 
demonstrations in education and to disseminate i nformat i on\deri ved from such 
activities; accompanying programs of teacher, administrator, and parent education 
are alsa the focus of the Laboratory's work. Programs conducted by the Labora- 
tory will offer a clear and firm prospect of being implemented by schools and 
other educational agencTes. In the course of these efforts, the" aim wi 11 be to 
assure that the eyal uat^tkoutcomes of research and development are effectively 
presented to schools and other educational agencies. - . - 

Thfe SouToeEipok is avai^abje fpr $3.95 from Far West Laboratory for Educational 
Research and Devel opn^nt , I $55 l^olsom Street , San Pranci sco, Cal i forni a 9^103. 
A check or money ordfer should accpmpany your order. California residents please 
prepay applicable s/4les tax. • ^' / 

Copyright (c) 1975 Jar LabdVa^ory' foV>^5l(iftiat iona 1 Research and Development. 

All rights reserved. C^opyright for these materials Is claimed only during tHie . 
period of development and evaluation, un less authbri z^atibn i s granted by the 
National Institute of Education to also claim copyright on the final materials. 
For information on the statufe of the copy r i ght' cl a im, contact the National 
Institute of Education. ( ^ 



r 

/ 



CONTENTS 



INTRODUCTION - , , 

ORIENTATION 3 

PROGRAM REPORTS ./ . 7 

' r 4 ^ 

Summary Chart . . / / >f 8 

Reports : / '/ 

Project Le^ership . , . . j 10 

Ernstspie/. . I3 

I ns t rucjnpna 1 Devel opment I ns t i tute 17 

Sett Goals for Local Schools: 

Ev^uator Training Program \ , 21 

Educational Goals and Objectives 
/A Model PiyDgram for Community and 

. / Professional Development 2k 

/Determining Instructional Purpos-es: 

/• A /raining Package ^ ...... . 28 

Interpersonal Communications 32 

Prentice-Hall Teacher Competency 

Devel opjnent System ' r 38 

Research Uti 1 izing Problem Solving (RUPS) /♦3 

Ecfucation for the People ^ Ijg 

Evaluation Workshop I: An Orientation 53 

Designing Instructional Programs , . 56 

CSE Elementary School Evaluation Kit: 

Needs Assessment 59 

Evaluation^for Program Improvement 62 

Shared Decision Making 65 

TRAINING RESOURCES 7I 

Purpos i ng . . ; 1 72 

^ Program Planning 1 75 

Program Development 1 78 

Program Evaluation . \ 80 

Organization for Problem Solving \ 87 



V 

INTRODUCTION 



What 16 I ns.tructlona 1 planning 
and management? What training pro- 
grams and resources are available to 
develop staff In these areas? What 
are the maJoF characteristics of the 
''avallabje products? Which ones seem 
to"be best suited fo local training 
^ needs? 

This SouvpeBook should begin to 
help you answer such questions. It 
Is deS^Igned f^r all who have a stake 
In selecting and conduct i ng Instruc- 
tional plannfng and management 
training activities — teachers, 
department heads, prrncFpals, curric- 
ulum specialists, assistant superin- 
tendents, and superintendents at 
preschool, elementary, and secondary 
school levels. This S(7uraeBook v^as ' 
developed to provide school personnel 
with efficient and effective access 
to Information on available training 
products. The SourceBdok should^ be 
viewed as a resource to Increase your 
awareness and unders tahd Ing of new 
products^-produ^p^ which train 
personnel In setting program goals, 
planning for development, and evalua- 
tion. 

There are three major sections 
In this SduroeBook . The first 
section, "Orientation," presents a 



view of Instructional planning and 
management held by the Far West Labor- 
atory . -Since the late 1960s, the 
staff has been involved in identi- • 
fying the skills and devel opi ng the 
ffaining needed for school personnel 
to plan, manage, and improve account- 
ability for instructional programs. 
The first section also suggests 
certain -guldel ines for reviewing and 
selecting materials to train person- 
nel in instructional planning and 
management . 

The second section, "Program 
Reports," describes 15 program 
packages that can be implemented 
within your school district to meet ' 
your specific needs for staff develop - 
ment or training . Each -report 
* should provide the decision makers 
with .enough information to decide 
whether or not to preview the pro- 
gram. 

The third major section , 
"TraTning Resources," p^resents an 
annotated list of 27 supplementary 
material s . The brief descriptions 
of various materials may help you in 
selecting resources to support or 
complement a training program In 
instructional planning and manage- 
ment, * r 
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ORIENTATION 



Instructional planning and 
management Is an area of educational 
administration concerned wl th provid- 
ing educational opportunities. 
Among the various functions of educa- ' 
tlonal administration Identified In 
the 1 1 terat ure--management of 
student learning, personnel, commu- 
nity relations, funds, and faclll- 
i ties*-- it is the management of 
student learning which has the most 
powerful and direct Impact on 
students. All, other functions can " 
be seen as support activities to 
this central function. 

In the schools, educational 
opportunities reach students In the 
form of Instructional programs. 
Instructional planners and managers 
are respqnslble for preparing these 
programs and must make a series 
of decisions. These personnel must 
determine Instructional purposes and 
design and evaluate Instructional 
resources and experiences. Deter- ^ 
mining the purposes of a* program 
lays the founddtlon for ^ound deci- 
sion making about Its design. 
Evaluation assesses ho^; well a 
program Is meeting Its specified 
purposes and, alternatively, whether 
thosa purposes require restatement. 
Effective decision making In both 
planning and evaluation stages 
depends on conditions within an 
organization that support group 
planning efforts. 

Purposing and Plannlnc^ 

Each dqclslon-maklng function 
Involves a number vof activities. 



Determining Instructional purposes 
Is' the Initial planning phase . At 
this point, a school's professional 
stdffj Its students and community, 
or a combined group set the 
priorities for Instruction. For 
example, a school's professional 
staff might develop ajist of clear, 
reasonably specific goals which, In 
turn, would be reviewed and ranked 
by the community. The process of 
setting goals and assessing needs 
provides focus and direction for 
future program planning. Program 
analysis Is another priority- 
setting activity, 'Systematic 
analysis allows an Instructional 
planner to. designate the source, 
nature, and scope of Instructional 
problems and the order In which they 
will require action. Having es- 
tablished goals and determined when 
will be the appropriate times to, take 
action, a school can further clarify 
'its purposes of Instruction by 
deriving objectives. 

Program Development 

Prjogram development Involve^ 
selecting and orqanlzlnti learning 
experiences and r^esources Into a 
cohesive whole . Activities during 
this phase Include selecting 
curricula and Instructional methods 
that suit a school's goals, and as- 
sessing possible alternatives In 
light of time, money and personnel 
resources arvd limitation^. Whether 
previously designed materials and ' 
methods are adopted or adapted or 
new approaches are Invented, measur- 
'Ing the proposed program against 



local needs, Qondltlons, and in- 
structional objectives Is integral to 
developing a good program. 

Evaluation 

Ideally, evaluation should begin 
when the purposes of a- program are 
estab-Ushed and should continue 
through the development phases . 
Procedures for later program moni- 
toring can be Incorporated at the 
planning and design stages. Once an 
instructional program has been'' 
selected and put Into operation, 
evaluation becomes Increasingly 
important. Through wel l-pl ar\ned 
evaluation^ one can assess the 
adequacy of program design, opera- 
tion, and goal achievement, deter- 
mine costs, and specify needed 
adjustments . 

Group Skills 

In today's school systems, most 
decisions are made by more than one 
person, and areas of responsibility 
frequently overlap. Numerous 
people may contribute to decision 
making, and numerous others are 
affected by the decisions made. In 
planning Instruction, principals, 
curriculum special Ists, teachers, 
superintendents, and assistant super- 
intendents frequently work together. 
Sometimes other groups such as 
school board members, community 
representati vies, and students are 
also Involved. Any acti vl ty Involv- 
' Ing a number of people requires 
application of group communlcaTlon 

ski iis > ; 

Selecting a Program 

The administrator who Is 
Investigating staff training In 
setting goa^s, program development, 
evaluation, or group skills will need 
to consider several factors. First, 
current educational practices and 
outcomes should be Identified. In 
doing so, one may determine existing 
needs, problems, capabilities, and 



limitations. It may also be help- * 
ful to analyze what t^pes of declslop 
are being made and by whom. In this 
manner, one may Isolate problems to 
be addressed by training and 
determine who'should receive 
Instruction. Other Important A:on- 
slderatlons, of course, are tlW and 
money. Possible sources of funding 
should be Investigated. Before 
settling on a choice, the administra- 
tor or other decision maker select- 
ing a training program should make 
sure that his or her staff have 
sufficient time available to 
receive the Instruction. Finally, 
the effectiveness of the program 
or materials at other sites should 
be Investigated. This might 
be accomplished by obtaining 
field test information from the 
developers or by obtaining names 
and addresses of previous and 
current users and contacting 
them. ^ 

Selecting among alternative 
training programs and resources, 
therefore, requires careful analysis 
of several factors. First, the 
probability that the product will * 
help to meet training or staff 
development goals must be determln(i.d. 
Second, It must, be decldex( whether ' 
the product applies to the^iieeds 
content Interests, skill levSels, 
values, and learning proces$« of the 
staff to be trained. Third, the 
feasibility of Implementing the 
training product In terms of avail- 
able tflme, cost, personnel, supplies, 
facilities, and equipment must be 
ascertal ned. 

Criteria - . ^ 

The programs and resources 
described In this SouroeBook were 
selected carefully to Insure that 
they are relevant to and useful f9r 
training In Instructional planning and 
management . Each t ra I nl ng program 
or resource was Included because It 
met all of the criteria for selec- 
tion. The criteria were as foPVpws: 



1. The intent of the product must 
be to provide or support train- 
ing act i vi ties. 

2. The training must be designated ' 
for middle management school 
personnel at preschool, 
elementary, and/or secondary 
levels whose primary fundtibn 
may be jfa) teaching, (b) school ^ 
administration, (<:) curriculum 
development, or (d) district 
administration. 

3. The content of the training 
program or training support^ 
tool must be in instructional 
planning and management. 

The product must be available for 
distribation on a national scale 
and/or repMcable through use of 
guidelines, information con- 
sultants', site visitations, or 
other comparable means. 

5- The product must have been 

placed on the market within the 
last f i ve years . 

The Far West Laboratory does * 
not intend to evaluate or endorse* 
any of the products described in this 
SourocBook. Inclusion of the prod- 
ucts rests exclusively oip the - 
criteria listed above. ' 

The most suitable of the train- 
ing programs or resources described 
in this SouvoeBook shouldbe selected 
within the framework of the school's 
own commitments, resources, and 
St ructure. 
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' PROGRAM REPORTS 




Descriptive reports on 15 
training programs are presented in 
^ this section. The reports are 
intended" to inform you about 
products which are now available 
to equip school personnel with knowl- 
edge and skills in instructional 
program planning and management. 
Each of the products • i nvol ves its 
users in "active'* learning, that 
Is, performing, practicing, or 
applying ski 1 Is or knowledge. 

Most of the 15 products provide 
training In theiarejas of problem 
' identification; hee^ds assessment, 
goal setting, or prograrn planning. 
Several focus on instructional pro- 
gram development artd Implementation. 
A .few stress program evaluation 
.skills. And several focus on group ^ 
communication skills, which are 
often Integral to effective program 
planning and decision making In 
schooj^' 

The 15 programs vyare selected c^n 
the basis of criteria developed at the 
Far West Laboratory for Educational 
Research dnd Development and listed on 
page 5. More than I50 products weYe 
Identified and considered during the 
selection process. Many were elW- 
Inated because their subject fS^cus 
did not fall clearly within the scope 
of Instructional planning and manage- 
ment. Some failed to provide 
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well-defined training activities; 
others were aimed at i napprpprl ate 
levels of target audience. Several 
met all of the selection criteria, 
but el ther" were not ready for pub- 
lication or distribution or could 
not be reviewed In sufficient detail 
to prepare reports for this edition 
of the .Source-Book. ^ 

Exclusion of av product does not, 
by any means, constitute a negative 
evaluation. Inclusion of a product, 
on the other hand, does not con- 
stitute endorsement. The 15' final 
Selections are n6t vl^ed by the Far 
West Laboratory as the 15 **best*^ 
programs In the nation. They do, 
however, meet all selection criteria 
and Illustrate the range and types of 
training which are currently available 
In the areas of Instructional planning 
and management . 

The chart on pages 8-9 organizes 
key Information- about all 15 of the 
programs so that you can review and 
compare their major features quickly 
and easily. The reports on pages IO-69 
provide more detailed Information 
about Individual programs— goals and 
objectives, training strategy, 
content of materials, etc. --as well 
as a comment from a Far West Labora- 
tory reviewer about the value, quail- ' 
ty, or possible Impact of each 
program. 
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r^ntrnt F<,.*u;i Hnstructional program 
planning and staff development. 

'^!vr^' c^f rvi^dwH One- to three-year, 
in-depth v^orkshop training. 

Aval iil^f !itj^ Now. 

Asj^oci^atHon of California 
S c hoo 1 Adrn i n i s t rti tors 
(ACSA) 

' 1575 Old Bayshore Hwy. - 

Burllnqame, California 
^ 9^010 I 

IHatvibutvy Same. In conjunction 
with Educational Management InstLfc'ute, 
c/o ACSA (address above), the program 
is avallable^n otber states. 



* Ta rge t A udje ncg . 

Project Leadership (K) provides 
training for district administrators 
and superintendents In the follorjlng 
management skills; confidence that 
others can make decisions; tolerance' 
for diversity; respect for competences 
of teachers; ability to enable deci- 
sions to be made at the lovjest possible 
level; and ability to provide a team- 
bulldlng environment. 

The Association of California 
School Administrators (ACS'A) provi-des 
the following Introduction components: 
needs assessment, problem solving, ad- 
ministrative support, and assessment 
ol ternatives; also Included are 
support materials such as a monograph 
called "Participative Management/ 
Decentralized Decision Making/Working 
Models" and an evaluation assessment 
instrument that provides the necessary 
background and information for identi- 
fication of the above management 
skills. Participants should also be 



able to ^pass along the skills gained 
to other's in their school or district. 



Goals and Objectives 
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PL 's goal is to provide in-depth 
training for district administrators 
that w] 1 1 provide them with skills and 
materials from a variety of agencies 
and sources. At the end of a three- 
year series of workshops and seminars, 
the participating team of tv^o or three 
district administrators will have Im- 
plemented In one or more schools in 
their district a f ie Id- tes t<id set of 
procedures. These procedures should at- 
tain at least one high-priority educa- 
tional goal of a selected school or of 
the entire district. I n te rmedi aWo^ jec- 
tives to arrive at this final goal are 
first, the identification of one or 
more top-priority goals to sGsTve as 
the focus for project activities; 
second, specification of a set of (a) 
planned procedures for attaining the 
goal, and (b) assessment procedu^ps 
for determining whether the goal\a^ 
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been met; and third, completion 
of field testing and evaluation of 
the planned procedures for arriving 
at the stated goal, and a description 
of modifications to be made as a re- 
sult of the field testing. 
• ' ; ■ 

The* t ralnlfig Is designed to en- 
able administrators to develop and 
maintain expertise as lieaders In 
reaching project goals and to deter- 
mine what skills and expertise the 
support staff (teachers , teacher 
aides , secretaries , custodians, par^ 
ents , stjjdents, community members, 
etc.) in their district need to fur- 
ther support the real Izatlon of 
project goals. Specifically, when 
the^Jv.admitii St r'ator has completed the 
prog^i^iam, he or she should be able to 
help^%he support staff to develop 
local evaluation criteria; Involve 
the 1 ocal . communi ty; gather evalua- 
tion data from the school; and 
determine the desired educational 
changes of local part ic i pan ts «^ 

Materials and Ec^uipment - 

- R-^ provide^ a system for change 
utilizing programs from a wide range 
of sources, 1 ncl udl ng agencies other 
than ACS A, programs . developed by 
staff within participating school dis- 
tricts, and ACSA-deve loped programs. 

The system begins with an Intro- 
ductory component booklet provided by 
ACSA that explains the program, it6 
basic assumptions- and goal's, provides 
a list of support agencies and In- 
cludes a ^0-ltem self-assessment 
Instrument designed to collect data 
regarding a person's understanding * 
of educational management -and to 
Identify the app ropr I ate eat ry level 
Into the ACSA training programs* 
ACSA stiff are' aval lable to make 
recommendations based or> the data- 
coll ected. Included in the assessment 
mstrumeat are subjects such as goal , 
setting -and deriving object i ves--who 
was Involve^ and what was the degree 
of atta'inment; partlcipatl\^e manage- 



ment and decentralization; Inter- 
personal communications; school cli- 
mate; assumptions about learning; prob- 
lem Solving and exam In lAg alternatives I 
monitoring and evaluating programs. 

Some programs offered during the 
series are the Research Uti 1 I zing 
Problem Solvinci (see page ^3) t ral n.l ng 
program and the Arlole planning com- 
ponent from Center for Educational 
Policy and Management; Shared De- 
cislorl Making Training from the Na- 
tional Cluster Coordi nati^on Center; 
Improving the School Climate from C.F. 
-Kettering Found^lon/Phi Delta Kappa; 
and the Far West Laboratory for Educa- 
tional Research and Development *s two 
unitSj Setting Coals and Deri vi ng 
Objectives . This list is extensive 
and continually evolving as research 
and development provides new tech- 
niques and programs that can be util- 
ized by Project Leadership . Two 
of the programs offered during the 
Pi workshops -- Determining Instruction- 
al Purposes and Shared Decision Making - 
are described on pages 28 and 65 re- 
spectively, of this SburaeBook. 

How to Use 

PL^ is a cooperative program, ^ 
Ideally for thtee years with each con-^ 
tract running for one year, for Cali- 
fornia school districts contracting 
with ACSA. The program is available 
outs ide.-of Cal 1 fornia in cooper- 
ation with the Educational Management 
Institute. 

Two- or three-person teams ,^ 
seldcted by their district represent 
It at two state workshops , regional 
training seminars', and four to 
eight satellite meetings per year. 
Participants are selected by the dis- 
trict on the basis of their expertise, 
as explained in "Target Audience." 
The seminars and workshops run three 
to five days, ^re org^pi^zed and conr 
ducted' by ACSA staff and outside con-" 
sultants, and offer as many as 20 
^different programs for participants . 
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to choose from. Prior to these. meet* 
lngs,^ACSA staff wMr help the Va*"" 
ticipant identify the areas ^raost 
suited to his pr her own needs and 
those of the school, based upon the 
data gathered in the assessment in- 
ventory (explained in '.'Materials 
and Equipment")^ ^ 

' ACSA presently has 55 districts 

represented in California. 
* . ^ * ■ 

The sites for^the workshops are 
determined by the density of partici- 
pants signing up within a given geo- 
graphical area, thereby cutting down 
on the traveling distance for the 
majority of participants. 

Trainee Assessment 



The majority of programs presented 
at tKe workshops and seminars include 
various types of instruments to measure 
change. For example, one program may 
provide specific pre- and posttests 
for trainees to assess specific skill 
gains; another program allots time for 
discussion and evaluation feedback. 
Each program provides its own specific 
form of evaluation. 

Product Development and Evaluati on 

PL , now in its sixth year, 
is an outgrowth of a similar, success- 
ful Project Eval^Sation program for 
elementary school administrators. 

Evaluation data i nd i cates positive 
results f rom PL^. The majority of ad- 
ministrators return to their districts 
and institute programs they have been 
Introduced to during the workshop and 
semi nar se r ies . 

Before being added to IPL^, programs 
were taken tTirough a fairly extensive 
review process^! This review con- 
sists of evaluation by ACSA staff, a 
subcommi ttee of educators, or experts • 
in the field and extensive interviews 
with persons who have'^ part tci pated 
in the program or in field tests. 



If the combined-attitudes of staff 
and interviewees are positive, ACSA 
includes the program in l^ts of- 
ferinqs at the various workshops 
and meetings. The prog rani must con- 
tinue to show positive results based 
upon the ACSA evaluation .and ^audi t 
procedures. 

Cost ' 

The annual cost for a two-person 
team to attend the serines is $1,500;^ 
for a three-person team, $2,500. This 
pays for any and all workshops attended. 
Parti cip^ants mus*t pay'their travel, ? 
lodging, and personal expenses. • 

Cominent — 7 

Project Leadership would seem 
ideal for dist^ricts th&t need vast ^ 
thanges in thai r sys tem i n order to 
meet accounta)bi 1 i ty requirements or 
other problems. It would also be val- 
uable' for school districts that wish 
to involve parents and other members 
of the community in the process of 
change.^ 

In addition tQ the advantages of . 
interacting with administrators f r9n).- 
other areas, PL^ intj^duces participants 
to* a wide range of programs and pro- 
vides them with necessary toqls, skills, 
and 'information to return" and institute 
programs witViin theM ? district. 
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Content <^QauB Communication problems 

caused by group planning and teaching. 
^ ■ • 

Ti^ye of Produot Flexibly schegduled, 
small and large group training. 

A-Qailability September 197^. a 

Vei)^ lopev Fran Thieman 

Chet Bumbarger 
Center for Educational 
Policy and Management 
University of Oregon 
1^72 Kinpaid Street 
Eugene, Oregon 
97^01 

Dietributor Same . 



Target Audience 

Ems tspiel is designed for 
teachers and administrators at 
elementary grade level3 1-6, specif- 
ically those involve^ in group 
pfanning, team teaching, and differ- 
entiated staffing. It has alsobeen 
used successfully with groups such 
as boards of education, central 
office staff, and students at all 
levels fr(^ elementary to* univer- 
sity. Entry level into training is 
not specified. However, the mater- 
ials appear basic enough that anyone 
who is interested would benefit from 
participating in training. 

deals and Objectives 

The major goal of Ernstspiel is 
to develop skills In verbal , noh- 
verbal, and written communication. 

It seeks to increase awareness 
that communication problems exist, 
to promote group cohesiveness through 



a common experience, and to illustrate 
that both cooperative and what the 
product calls 'Meviant" persons are 
important in the group 'plianni ng 
process ^ 

Materials. -and Equipment 
# 

The Ernstspiel kft contains eight 
packets, each covering a specific 
communication skill. The fjrst six 
packets deal with the single concepts 
of "One-Way and Two-Way," "Tacit^" 
"Non-Verbal ," "By-Pass ," "Overload," 
and "Wri*tten" communications; the lasti 
two with "Group Norms" and "Individual/ 
Versus Grogp." The latter two require 
the participants to use the knowledge| 
and skills learned in the previous 
packets to solve more odmplex communl| 
cation problems. All packets take tne 
form o^f puzzles or games. The Center 
for Educational Policy and Managemer 
defines a puzz-le,^,as the individual 
fng only to the materials, and a gaf 
as a group being required to functi/on 
as a team. 
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Each packet has instructions 
for the leader, the necessary gaming 
and puzzle materfals (enough for ^0 
participants) , and master copies of 
participants' forms and worksheets 
for duplication, plus two cassette 
tapes and a filmstrip.- Each uses a 
different number of. parti cipants in 
the actual'^training, and the remain- 
der of the participants observe the 
procedures. The developers say the 
Ideal number /for training- is 30; 
minimum 5, maximum ^0. The leader^s 
manual give^ explicit instructions 
on how to organize -the group, con- 
duct the training, and question the 
parti-tipants in ways that will stim- 
ulate discussion and promote group- 
and self-evaluation. A summary of 
literature related to each communi- 
cation skill is also included, -along 
with a supplementg^ry information 
booklet for the leader covering 
typical quest ions and problems he 
or she Is likely to encounter, with 
suggested solutjons and answers. 
For example, clues to the solutions 
of the problems are sometimes fouftd 
In the reading, and the booklet tells 
the leader how the reading can be \ 
done to emphasize the clues. The 
booklet also provides anecdotes that 
the leader can tell In order to 
cl<ar}fy the exercise. ^ 

How to Use 

The (developers designed the 
materials\^ incorporate both auto- 
tellc activities (enjoyable in and of 
themselves) and elements of the concevpt 
of "ernstsplel" (where play Is taken 
seriously but has no lasting social 
consequence 1 f one falls to win). 

Ernstspiel is appropriate for work- 
shops, preservice and inservice train- - 
ing, staff and faculty meetings, and . 
staff development sessions. Far preserv- 
ice workshops It is recommended that the 
approximately eight hours of training 
be scheduled In no less than one week. 

Each packet takes approximately 



an hour to complete, and the devel- 
opers recommend that i^o more than 
two be presented per session. 
Although each packet is complete in 
itself and could be q^sed as an indi- 
vidual training unit, the packets 
were designed as an integrated set 
and should be p/-esented sequentially. 

The leader must read and under- 
stand all of phe materials. It is, rec- 
ommended that the leader have an ' 
assistant to hand out and collec^t 
papers so that he or she can be free 
for dfscussions and interaction. 

, The '^One-Way and Two-Way" commun- 
ication packet involves each partici- 
pant as a communicator or receiver. 
Ijhe communicator,* out of sight of the 
pceiyer, describes a geometric fig- 
Are, which the receiver then draws, 
lb 'one-way communication, the receiver 
cAnnot ask questions whi le drawing 
die figure. During the two-way com- 
nunication practice, the receiver may 
.ask questions of the commun i cator ^to 
further clarify what he or she is' to 
draw. Each person summarizes his/her 
feelings on either a receiver's or 
communicator »s questionnai re form', 
and then discussions follow to deter- 
mine ways of improving communication 
skills. 

"Tacit" communication usually 
occurs under constrained conditions 
and reljes on the existence of ^ 
specific bits of • i nformat i on held In 
common or inferred from th^ situation. 
For example, two people unable to com- 
municate with each other must antici- 
pate what the other would dp In a 
given situation. To practlbe this 
skill, participants form teams with 
each person on each team having an 
unidentified partner on theiOpposite 
team. Each person must then answer ' 
question*: and, given certairl informa- 
tion, solve problems according to his 
or her guesses about the responses of 
the unknown partner. 

In "Nonverbal" communication ^ 
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one person copies a simple drawing 
and then Msses it *along^for redraw- 
ing to another person, artd so on 
down the line. The group studies 
the distortions and changes that 
occur from the first to the final 
' drawing and discusses the weaknesses 
and advantages of nonverbal 
communication. 

•the "By-Pass" communication 
packet focuses on the error of 
treatiTW all individuals alike and 
demonstrates the feelings of an 
individual being by-passed. Par- 
ticipants attempt to explain to each 
..other various jargon, terminology, 
and statements and then discuss 
their success at it. Discussions 
center on the causes and effect of 
by-pass communication and how to 
analyze. when, if, and to what ex- 
tent you are getting your message 
across when speaking to an indi- 
vidual or to a group. 

"Overload" communication dem- 
onstrates the frustration of attempt- 
ing to absorb a large quantity of 
infarmation in a short time. Par- 
ticipants listen to a recording that 
Is overloaded with details and fig- 
ures and then attempt to convey to 
anpther person what was heard. 

"Written" communication requires 
each. part i ci pant to write a 100-200 
word description of a program of his 
or her choice prior to the training 
session. Participants identify the 
target audience to whom the writing 
is directed, then, with a formula 
provided, evaluate each description 
for its probable effectiveness in 
reaching the target audience. 

"Group Norms" deals with the 
reactions of a person toward the 
behaylor of a deviant member. Groups 
of five are formed for the purpose 
of developing lists of tasks that 
a teacher's aide would perform'. Each 
group must follow the same rules and 
constraints, such as total agreement 



between members, etc. An observer* 
for each group records on a form each 
time each member gives information, 
asks a question. Is positive or neg- 
ative toward another's ideas. The 
basic purpose ts to demonstrate the 
value of a deviant In a group* A dis- 
cussion follows tt> determine how well 
the groups worked together, .reactions 
to deviant members, and methods for 
working with deviants. 

In' the J'lndivMual Versus Group" 
packet, groups of five members each 
compete by playing a word game. 
Given a set of letters, members of 
each group must form different words 
^ using'specif ic letters. Each 
/ member has a different set of letters \ 
he or she must use to make words. 
Each i^ dependent upon the other 
members of the group to form his 
or her required words, because letters 
must be exchanged. The cooperation of * 
each member Is essential If the group 
Is to complete Its task. Discussions 
follow about the advantages, disadvan- 
tages, and productiveness of working 
together and individually. | 

Trainee Assessment \ 

The participants are able t\ 
' evaluate their progress through ttie 
discussions that follow each . activi ty 
in the packets. The questions pro- 
vided to the leader are aimed at eval- 
uating • individual performance and 
group functions. Each person Is asked 
to evaluate him- or herself, the group, 
and fellow participants. There are no 
formal test instruments. 

Product Development and Evaluation 

The Center for Educational Policy 
and Management is developing programs for 
six areas in which they feel training and 
information are needed: communication, 
pi anning, leadership, reaching agreements 
socializing team membership, and initiat- 
ing change. These areas -were determined 
by research and a heav^ and continuous' 
flow of requests for f(?lp from educators 
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a 1 1 over the country. , 

_ The need for Improvln/ communf- 
cation resulted- In the devfelopment of 
ErnstsDiel . The focus of /the mater- 
ials is determined by the/deve lopers ' 
belief that some form of /team teach- 
. ing, differentiated staffing, or 
multlunit operation wi 1 l/be the\ 
teaching method of the future. 
Ernstspiel was tested v/tth $00 people 
In six sites from Flor/ida to Oregon. 
The sites chosen represented a wide 
range of geographical areas . Pre- 
and post tests were dfeve loped by out- 
side evaluators to father formative 
and summatlve data. One Instrument 
looked at usability of the materials ' 
and at attitude change. Th^e secohd 
Instrument focused on structural 'ind 
behavioral changes. Pf^- and post- 
test scpres Indicate higher ^toSttest 
scores In social Interaction and grou"^ 
problem-solving ski 1 is. ' 

V 

Because Ernstspiel materials were 
designed to be self-Instructional, 
testing was carried out under three 
sets of conditions: (l) developers 
of materials at the Center for Educa- 
tional Policy and Management conducted 
Inservice training with work' groups; 
(2) Center-trained leaders conducted 
the programs; (3) materials were used 
by leaders designated by an outside 
.agency, the National School -Develop- 
ment Council. Without the Center for 
Educational Policy and Management 
training, these leaders conducted' 
sessions with Intact work groups . ""In 
addition, a fourth control group whp 
received no Ernstspiel traininn com- 
pleted the evaluation instruments.- 

Cost 




reaching Its goals of Increasing 
awareness of communication problems 
and" promoting group coheslveneas 
through a common experience. Bringing 
participants together for autotellc 
and ernstspiel activities (see 
"How to Use") creates a relaxed, 
open atmosphere we 1 1 suited to 
communication. 



The "Group Norms" packet makes no 
provision for role playing; hence, it 
will not reach Its goal of d4monstratlng 
the value of a deviant personality, un- 
less such a person is already in the 
group. The packet implies that, in this 
Instance, "deviance" suggests a more 
extreme position than simple disagree- 
ment with the. majori ty. 

Some of the exercises spem to 
emphasize a person's ability to memorize 
information, rather than his or her 
ability to communicate it. ' ' ' 

The "Individual Versi/| Group" 
packet appears to success fulily demon- 
strate both the difficulties of grt5up 
cooperation and the necessity for, ft.;, 
it requires the completion of t^sks 
that depend -on individuals cooperating 
for the sake of the group. 

%. 

Ernstspiel 's content and flexi- 
bility appear to make It a gooJ intro- 
duction to communication problems for^ ' 
those who do not know each other wel 1 ' 
and are about to undertake, .or may 
already be Involved In, group-oriented 
tasks; such tasks might be those re- 
quired of curriculum planning committees 
or decision-making groups. 



The kit is $9'*. 75 and includes 
all games and puzzles for ^jO persons, 
masters of forms for duplicatioh, and 
one copy each of the leader's books, 
one kit can be used many times. 

Comment 



Ernstspiel seems fairly good at 
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INSTRUCTIONAL DEVELOPMENT INSTITUTE (IDI) 



Content Fooue How\to develop an 
I ns t rue 1 1 ona 1 p rog ram . 

y?ypg of Product Five-day, multi- 
media workshop. 

Availability Now. 

Developer University Consortium 
for I nstructional 
Development and Tech- 
nology (ucidt) 

For a list of the UCIDT 
Directors see page 19. 



Distributor 



University Consortium 
for Instructional 
Development and Tech- 
nology , 
National Office 
Instructional Media Center 
Michigan State University 
East Lansing, Michigan 
^882^ / 



Target Audience 

The Instructional Development 
InstiTute ( IDI) Workshop Is Intended 
for teachi^r's , district level super- 
intendents, assistant superintendents 
of instruction, principals and 
assistant principals, school board 
members, and curriculum content 
specialists, at grade levels K-12. 
IDI cal Is this group TABS: teachers, 
administrators, board members, and 
specialists. The ideal TABS distri- 
bution is 

30-35 teachers (at least 2-3 

from each school represented) , 
i*-6 administrators (2-3 dis- 
trict level superl ntenden^ts 
or assistant superintendents 
and 2-3 principals or assistant 
principals). 
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2-3 school board members , and " 
^-6 specialists-curriculum 
contents areas , rrk^dla. ^ « 

J In addition, participants in the 
workshop should be strongly interested 
In improving th^i r schools and dis- 
tricts and be considered opinion leaders 
by their colleaguesV The developers 
believe these qualifications are ^ 
crucial to the prdgram* 

Goals and Objectives 

IDI has two basic goals: first, 
to motivate participants to want to 
Institute changes and new programs In 
their schools and districts; second i to 
provide training in the skills necessary 
for 'instituting chaliige and new programs. 



How to Use 

The m_ is a i»0-hour, 5-day 
workshop. The training follows a 
thr^ee stage model- 



The first stage, define , con- 
sisTis of conducting needs assessment 
and Identifying problems; analyzing 
current schoolyiondi tlons and pin- 
pointing target student audiences- 
and organizing staff into management 
teams to determine tasi<s , responsi- 
bilities, and timel ines. 

The second stage, develop . ' 
includes preparing behavioral objec- 

IV\ '^'"""9 appropriate teaching 
methods and instructional media; and 
constructing prototype (tryout) pro- 
grams to reach the objectives. 

, The third stage, evaluate, con- 
sists of testing the prototJ^pe pro- 
grams on specific student audiences- 
analyzing the results in terms of the 
original problem, the objectives set 
and strategies and media used; and ' 
reviewing the steps in the process 
aad deciding whether to implement 
the new programs or make further 
revisions. 

Training is divided into seven 
contiguous, related units. U^iit 1 ' 
'The Context of Instructional Devel- 
opment," and Unit 2, "Introduction'" 
to the Instructional Development 
Process," introduce the participants 
to current Innovations In education 
designed to strengthen existing pro- 
grams. This is accomplished through 
a number of films and slides of case 
studies and participant handout 
materials. 

t 

Units 3, "Innovation-Interaction 
Game," and i», "Problem Definition 
and Organization," provide training 
in all of the define skilU. Unit 3 
IS a simulation which sensitizes the 
parti cipantis to human needs and 
problems which appear whenever change 
Is undertaken In a* School district 



- A workbook called Application of Gen- 
eral Systems Thenr y to InstruFET?^;^ 
Development Qive^ th^ — 
heuristics for avoiding such problems. 
This workbook also give? participants 
further skills In problem definition 
and management organization, the 
topic of Uhft'"'*. Other Unit k activ- 
ities Include sUde/tape presentations 
emphasizing the. need for systematic 
Instructional design. The goal of 
Units 3 and i» Is to commit partici- 
pants to systematic analysis as an 
approach to problem solving. 

Units 5 and 6 are designed f<^r 
participants to practice the develop 
skills. In Unit 5, "Spec I f I cat I on of 
Objectives," participants complete 
a booklet entitled What You Alwavk 
Wanted to Know About Performance 
Objectiv es but Were Afraid to Ask , 
a^'play a game entl.tred "Objectives 
Marketplace Game," both of Which 
are designed to give practice In 
writing objectives. Following these 
two exercises, participants write 
behavioral objectives relevant' to 
their particylar Instructional 
situations. In Unit 6 ,* "Sped f I ca- 
tion of Methods," part I c Ipantsi learn ' 
to select the appropriate teaching 
strategies and Instructional mkdia 
to meet the objectives they ha^e 
developed. The manual which accom- 
panies Urtit 6 Is Selecting Instructiona l 
Strategies and Medii: This manUl and™ 
all others Include examples, procedures 
for group and Individual planning exer- 
cises, and brainstorming suggestions. 

Unit 7, "Evaluation and Impjle- 
mentatlon," consists of a media pre- 
sentation correlated* to exercises In 
a workbodk called Evaluation for 
instructional Development . The~wo rk - 
book exercises are to be completed 
based on Information stated or Implied ' 
In the fllmstrlp. The workbook also 
Includes an evaluation reference sec- 
tion for the pci/rtlclpants to use after 
the workshop \i over. 

The Institute concludes wj th a 
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mul 1 1 media presentation frevlewlng 
the instructional development pro- 
cess and emphasizing the ultimate- 
benefits to students, tK<p ^jpctiool , 
and the community when instructional 
development is used effectively. 

The IDI Is^eslgned s0 that 
pant I CI pants learn about the process 
of instructional development as they 
work on a real problem facing them 
In the classroom. 

The IDI Is conducted by a trained 
team of four persons- for a group of 
50 participants. This team can be 
provided by any one of the five 
institutions; listed below: 

Dr. Charles F. Schul ler, 
UCIDt Director 
' National Office 
lnstruct^onal Media Center 
Michigan State University 
East Lansing, Michigan 
4882^ 

-Dr. William Allen 
Department of Instructional 

Technology 
Room 801 , Phi 1 1 ips Hal 1 
University of Southern 
^ Gal I fornia 

Los Angeles, California 
90007 

Dr. Jack Edl Ing, V. P. 
United States International 
University 
10^55 Pomerado Road 
San Diego, Cal I fornia 
92131 

Dr. Donald Ely, Director 
Area of Instructional 

Technology 
Syracuse University 
130 Huntington Hal 1 
Syracuse, New York 
13210 

Dr. Mendel Sherman 
Department of' Instructional 
System Technology 
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Mitchell Hall 
Indiana Unlversi ty 
Bloomington , Indl ana^—-^ 
^7^01 f X. D 

If desired, a larwe distrlcijcan 
have a team of Its ownVt ral ned--i/ 
rather lengthy and exp^y^slve fok/r-step 
process — to provide I ns t rtiQyj0^to 
other members of Its district yfetaff . 
Such training wou 1 d requi re £4 ) attend- 
ing an IDI workshop as a pa/t I cl pant ; 
(2) attendl nga^pecl al^^x^re 
workshop fof IDI tflreptors; (3) assi'st- 
. Ing at an IDyl coruiug^d by an experi- 
enced dl rectoV^^^td {k) directing^an 
IDI wl th an experienced director 
assisting. 

Trainee Assessment 

Pro'^ress In the Institute Is 
assessed through validated Interest and 
attitude scales, four achievement tests 
and a Prototype Specification Exercise 
workbook, which is* evaluated by the 
l^nstitute staff and the participant's 
peers* Evaluation Is provided by the 
institute staff in three follow-up 
' visits 3-5, 8-12, and 16-2^ weeks after 
the workshop Is over. Summatlve evalu- 
ation data enables the consortium staff 
to evaluate participants' on-the-job 
performance and provide any additional 
assistance needed to implement 
Instructional development practices. 

Self-test exercises at appropriate 
points throughout the workshop provide 
the participants with feedback regard- 
I ng I ns truct lonal deve lopment Qoncepts 
and procedures. Group activities and'' 
discussions also allow the participants 
to observe other teams at work. 

Product Development and Evaluation 

The Instructional Development 
Institute was developed by the Unive r- 
Sity Consortium for Instructional Devel- 
opment and Technology (UCIDT); tlie mem- 
bership Is listed under "How to Use." 
Each of the units of the I/O I was Individ 
uajly field tested and revised and then 



tested together in four prototype 
institutes during 1970^72. This pro- 
totype testing was done with 50 TABS 
participants each time and was com" 
'pleted- in school systems in Detroit, 
Phoenix, Georgia, and Minneapolis. 
By December 197^, approximately kOO 
I D I ' s reaching nearly 20,000 teachers 
had been held in school systems 
across the United States. In 
addition, about 70 agencies have been' 
trained by UCIDT to conduct these 
workshops. 

A UCIDT study pf sehected 
school districts in August I973 
showed that IDI was successful in ^ 
motivating 80 percent of the partici- 
pants to generate feasible plans for 
instructional improvement. 

Costs . 

The materials for an IDI which 
trains 50 staff membiers can be pur- 
chased for $1,080 from the distrib- 
utor. Trainer costs are $2,^00, plus 
travel expenses and per diem, for a 
four-person team to come to your 
district for five days. ^ 

The school district is asked to 
provide meeting rooms, projection 
equipment, a projectionist and a 
small amount bf secretarial and 
duplicating assistance. 

Comment 

The strong motivation and Inter- 
est prerequisites for participants 
give the IDI the possibility of 
being an exciting workshop. 

An Important aspect of I D I is 
that participants are "turned on" to 
want to solve district problems. 
When I Dl ts completed,, a core of 
trained personnel, who will seek to 
Ijnplement the plans for change gener- 
ated during training and seek out 
new s.olgtlons to problems, should 
aval lable. 



SETTING GOALS FOR LOCAL SCHOOLS: 



Evaluator 
Training . 
Prog ram 



Content Poaus Goal setting. 

^ype of Produat Training In goal- 
setting ski lis. 

0 m 

Availability ^ 

Developev James M. Morgan 

Department of Research 

and Development 
Cincinnati Pub) Ic Schools 
Cincinnati , Ohio 
^5202 

Distributor Sbme, 



Target Audience 

This program .is In-tended for 
school system evaluation specialists 
or central administrative personnel 
with a background In evaluation, such 
as evaluators assigned to local 
schools- It Is recommended that 
partIclFE£)nts should also have had 
graduate academic work In tests and 
measurements, Introductory statistics, 
and sampling and survey methods. 



Goals and 



Objectives 



The training process Is Initially 
aimed at |local school situations; The 
evaluators trained In this program 
will acquire the following skills: 
Identifying the needs and problems of 
a school; ctross-val Idating these prob- 
lems .with objective data and community 
surveys; translating high priority^ 
problems Into goals foi^ the school - 
community to pursue; and assisting the' 
local school -community groups In con- 
ducting the goal-setting process for 
thel r schools . * 

^, ■ . - 

The program depends on broad- 
based community Involvement fn^The ) 
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school. Using the personal cfoncerns of 
the school-communi ty groups a"s part of 
the training process, evaluators who 
participate In this program will be 
able to provide technical assistance to 
these groups In setting goals for their 
schools. 

Materials and Equipment 

The Evaluator Training Program is 
organized into 11, self-contained 
instructional units of varying lengths. 
Each \jnit covers c)ne aspect of the goal- 
setting process : 

'•Community Participation In LocaJ 

School Goal Setting''; 
*'A Process for Local School Goal 

Setting"; 
"Group Techniques for Identifying 

School Problems"; 
"Analyzing .and Clarifying School 

Problems"; 
"Using School Data to Identify 

and Vall(date Problems"; 
"Presenting SchopI Data 'to School- 

Communi ty Groups"; 
"Surveying Community Opinion: 
Constructing and Administering 
a Community Survey"; 
"Surveying Community Opinion: 



21 



Interpreting Survey Results"; 
"Translating Problems to GoaU";' 
"Placing School Goals In 
/ Priority"; and 

"Reducing School Goals to 
Objectives ." 

Slide/tape presentations are 
used throughout and^are important to 
the success of the tVainlng; they are 
follcxved by readings on the subject 
presented, exercises based on what the 
the participants have^een or read, 
or both. Easy access to a slide 
projector, cassette tape recorder, 
and projection screen Is therefore 
necessary. 

The trainer's manual contains 
detailed i<;is truct Ions for organizing 
and conducting a workshop with thi^ 
material. The first section gives/ 
a complete description of the train- 
ing program, including the purposes 
and assumptions behind Its develop- 
ment, the content of each unit, and 
how evaluation of the training Is 
conducted. The second section con- 
tains specific Instructions on how 
to conduct each unit of the training 
program. The third section h^s the 
training materials themselves. 

How to Use 

The training materials "have .been 
constructed so that the program Is, 
largely sel f- I ns t ruct lona 1 and the 
trainer Is primarily an Instructional 
manager. The trainer ovei^sees the 
preparation and sequencing of mater- 
ials, coordinates the performance of 
exercises, and assists In discussions. 
The developers recommend that the per- 
son selected for this role have some 
experience In managing group 
training sessions; training In ed- 
ucational tests and measurer^ents and 
sampling and survey methods; and a 
background fn school -commun I ty • 
relations. 

All the participants do the 
exercise on group decision making tn 



Unit I, which provides the experience 
of making declslonsi by consensus rather 
than Individually. In Unit IJ, the 
goal-setting process and format of the 
training projjram jre described In a . 
slide/tape presentation; afterwards, 
the participants are asked to fill out 
an Entrance Skills Inventory, which Is 
an Inventory ^of their goal-setting 
skills." The resul ts of the I r responses 
dictate the structure of the training. 
For example, if all or most of them are 
well experienced In a skill that Is 
covered In a particular unit, then all 
or part of the unl t wi 1 1 be omi tted at 
the discretion of the'tralner. The 
sequejnce of the units might alsQ be 
changed, and the developers have 
specific recommendations when change 
becomes necessary. ^ 

Because the units vary In length 
and require different amounts of time * 
for completion, the trainer has to 
determine the best way to program 
them. They can be done In three full- 
day workshops, five ha 1 f-day work- 
shops, or be scheduled once or twice 
a week. The- deve lope rs recommend 
, that the I ndl vi dua 1 un I ts not be 
, scheduled more than one week apart. 

* 

The program will work best 
with a minimum of 8-12 persons 
and a maximum of 32. At certain 
points, subgroups of ^ or, less must ^ 
be used. They are given exercises, . 
Individual worksheets, or as a small 
group they assume the roles of parent, 
teacher, student, of staff; after 
completing these assignments, 
the small groups Join for further^ 
d I scuss I on . 
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Trainee Assessmeni 

Evaluation Is based on Immedlajte 
feedbacks The participants may either 
read a printed passage or look at a 
slide/tape presentation. After doing 
an exercise based on what they saw or 
read, they receive immediate knowledge 
of their performance from suggested 
answers or ]n-group discussions. In 



addition to self-evaluation, there is 
a pre-post self-rating scale which 
shows participants their growth in 
the skills coveVed in^ the program. 
The Entrance Skills Inventory taken 
at the outset gives participants an 
opportunity to see at what level they 
begin and to make a statement about 
their expectations for the training. 
The Exit Skills Inventory is com- 
pleted at the end of the program. By 
comparing the two skills inventories, 
the participants see their progress 
and areas of weakness. They can 
review the units covering those areas 
that need strengthening. 

Product Development and Evaluation 

The tincinnati Public Schools 
began work on the development of 
these training materials in 1972 with 
a grant from the National Institute 
of Education. An initial set of 
materials was pilot tested in the 
winter of 197^ with a group of ten 
evaluatron and administrative per- 
sonnel from the Cincinnati Public 
Schools-r Based oa this pilot test, 
a second version was produced and . 
field tested with 25 guidance coun- 
selors in Cincinnati during the spring 
of, 1973-.^Epch group completed a form- 
ative evaluation which asked for 
ratings of the process and content of 
the unit and suggestions for^lmprove- 
ments. Copies of the materials were 
also reviewed by^-p^^e^e*^ona 1 educa- 
tors„^n ie)/aluation or community in- 
volvement. , The Evaluator Training 
Program is the result of those, re^^aws 
and tests. 



systematic approach to change in 
local schools. After goals have been 
set, programs are j^lgnnecl to 'reach the 
goals, and then the programs are im- 
plemented and evaluated. The partici- 
pants should be reminded that the 
program Is limited to the development 
of clear and valid goals as the first 
step 1 n local school program 
development . 

Tl^e product related to this pro- 
gram is Setting Goals for Local 
Schools-Community Training Program . 
As the title implies, it is intended 
to train local school principals, 
parents, and members of school- 
community organ izati-ons in goal- 
setting sklHs, and follows a 
format similar to the Evaluator 
Training Program . 



CcfSt's 



Trainee's materials - $5.00 
(each) 

Trainer's Manual - $17.00 (In- 
' eludes cassette and slides) 



Comments 



The program assumes that goal 
setting Is the first step In a 
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EDUCATIONAL GOALS AND OBJECTIVES 



A Model Program 
for . 

Community and 
Professional , 
Development 



Target Audience - 

* . The program Is meant for district 
superintendents and district boards' of 
trustees looking for a plan and mater- 
ials to Involve the cgmmunlty, schoo-I 
staff, and students fn determining 
district goals and translating them 
into object I ves. Among the persons 
the superintendent or board might wish 
to Include In the goal-setting sesjSions 
are parents of school age children; 
vrepresentati ves of school -aff 11 iated 
organizations; ethnic and socio-^ 
economic groups ; government organiza- 
tions; business, service, religious, 
anxJ cultural organizations; school 
board members, administrative staff, 
classroom teacher's, and certified 
and classified personnel. The pra- * 
grammed course in writing objectives 
IS Intended for te'achers of ail grades 



Content Foquq Strategies for develop- 
ing community-based educational goiils; 
training for professional staff ip 
writing performande objectives. 

" Tiwe of PvoduQt Both group materials 
for community goal -setting sessions 
and self-instructional workbooks, 

^Avai lability Since 1971, 

Developer Kehth Rose ' 

Northern Cai Ifornia 
P'rogram Development 
Center 
AJH 100 

California State UnTversity 
, Chico, Cai 1 fornia ^ 
' , 95926 



Distributor 



Phi Delta Kappd 
Eighth and Union Streets 
Bioomington',^ IndianxJ 
^7^01 

ft 



and a 1 1 subjects ' 

Goals and Objectives 

The program v;as 
mine the educational 



des 1 gned 
goals of 



to deter- 
the 

community and develop program objec- 
tives In keeping' with them. Materials 
are included for inservice t(B^cKer 
training in writJng the program objec- 
tives. Upon compietincjwthc tr^ainiog, 
the participant should Be abia TT>^dis- 
tinguish goals' l^rw object i ves , identify 
the parts of a performance objective, 
and recognize' and write three types an<i 
levels of objectives . 



The fpl i'0^;ing materials are pre 
vided fcMT planning and conduqting 
.community goal -ranking session^, and 



2n 

2i» 



for training staff tovrite objec- 
tives for selected programs. 

^ An Administrator's Manual ; 
Educatioaal Goals and Objectives 
details, alternative methods for 
selecting a commilitee of •community 

^ representatives to rank district 
goals. It also suggests h'pw the 
goal-ranking sessions might be 

'conducted and how the trainihg 
might bif^ organ! zed. Sample news 
releases explaining the prograjn and 
sample letters asking community 
representatives to participate are 
Includfed.. ^ 

Goal -ranking materials are * 
provided for 60 people, the recom- 
mended number of committee members. 
Tftfcse Include display boards, goal 
cards, discs, goal sheets , di recti oi>s ^ 
summary and t^tHy sheets, and in- 
struments for^rating current school 
p^-ograms. Sample copi es ; wh Teh the 
^district can' reproduce , are also 

available in Spanish. 
■ ^ 

•The * i nservice tralnipg program 
consists of both individual and group 
activltsies. The part i ci pant *s work- 
book , A, Programmed Course, fpr the 
Writing oft Performance Objectives , 
contains sequential frames and exer- 
cises'l^n writing objectives. These 
exercises are to be completed between- 
meetings. The accompanying I ns true- 
tor's H^nua I , in addition to provid-^^,^ 
'i<ig guidelines for conducting the 
course, 'Suggest^ group exercises. 
Group' sessions with the instructgr 
are a-lso an occasion for explanation, * 
review, and discussion. The sessions 
can be conducted in four weekly meet- 
ings, for ^ total of eight hours of 
group instruction. Other features of 
the manual are a section on gtaidelines 
^or organizing a program to develop 
objectives for each school ,^ recomT 
njepdations for establishing these 
ot^'ectives in ^he school, and a TOdel 
foTr developing instructional sequences 
to meet the objectives. . 



• Two i terns of equipment are rec- 
ommended: ^.an^verhead projector to be 
used duVing^the training cours§ and an 
adding machine or calculator for the 
tally clerk*' who will compute the results 
of the goal ranking. 

An audictslide orientation presen- 
tation which may be borrowed from the 
distributor free of charge,^ also re- 
quires a projector and cassette 
recorder. 



How to Use 



/ 



The time span for the full 
gram, from tl>€ first task force 
to the presentation of objectives by 
school staff \is six months to, one year 



pro- 
meet i ng^ 
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Time needed in planaing and prepar- 
ation is approximately eight to ten 
weeks, including planning sessions and 
the selection of commi t tee members . 

The two community goal -ranking 
meetings take place about a week apart. 
The firs't meeting takes three hours; v 
the second, two hours. Between ^0 and 
60 people shouTd represent ti}e commun- 
ity. Then, in groups of four, members 
attempt to reach" a consensus on the 
- ranking of the goals. After all the 
smal l groups have (reached an agreement 
through discusision, the average for 
e_ach goal score, ys^^xrompt^d and the 
results are repq^rted to the merpbers. 
One week later the committer meets 
again to. assess how we 1 I thA districtis 
current programs are meet ing\ the goals. 
Wi.th the assistance of scTiooV staff 
and students, the programs are rated on 
a scale* and^the results computed. 

School staff members, either 
teachers or administrators}^ are needed 
to swerve as monitors for each small 
group. Their function is to assist the 
group and clarffy procedyres whenever 
necessary. Prior to the first meeting, 
a half-hour training session Is held - 
for the monitors. A taj^ly clerks is 
^;;^quired to compute results at the 



meetings aricl ibould^be provided with, c 
calculator or an adding machine. 

A cadre of teachers experienced 
in wrfting performance objectives is 
selected to conduct the training 
course. These instructors review 
the program materials and set the 
schedule of instruction for a deter- 
mined number of the district's 
teachers. 

The" training course requires 
four to eight hours of meeting time. 
If the part i ci pants meet once a week 
for two hours, the course can be 
completed in four weeks. In addition 
to attending the meetings, partici- 
pants complete frames and exercises 
between sessions. One instructor is 
required for every 20 participants. 

Actual development of program 
level objectives for the d i s t r i ct * s 
schools requi res from two to six 
fhonjths , depending on the size of the 
district. Guidelines are provided 
i n )the materi als . 

Trainee Assessment 

# ■ vt 

The programmed training materials 
allow continuous assessment. The 
participants complete the frames and 
check the correctness ;0f their own 
responses against the answers In thej, 
workbook. The instructor reviews^ 
those .activi ties for which there are 
no predetermined "right" answers, 
i.e., the objectives the participants 
write. Group discussion is an addi- 
tional source of feedback. , 

Product fevelopment and Evaluation 



The three units 
Goals and Objectives 



of the Educational 
A Model 



Program 

For Community and Professional Devel- 
opment were developed sequenti al ly and 
field tested concurrently at various * 
sites in Northern California. 

Educational Goals and Objectives 



was revi sed and 



field tested in 15 
/ 



school districts, in Northern California 
between September 1971 and May 1972. 
A factor analysis of tKe results of 
goals-ranki n"g was accomplished to 
verify the independence of the 18 goal 
category statements. Between 
October 1971 and 'June 1973, over 76 
school districts and 2,^98 individuals 
implemented the model program. * 



f 
I n 

in i mp 1 emen tat i on '-of 
Educational Goals an 



A Programmed Course for ^the Writing 
Performance Obiectives wa!^ developed 
fall -winter 197^^'tfe=5i4S5i s t educator^ 

haseTrtMa^ the 

Object i ves ntocteL 

(performance objectives) . The coUr$e , ' 
content was critiqued by a group of 15 
administrators 1n February 1972 and 
tften was extensively field tested and 
revised during March 1972-January 1973 in 
18 Northern California school districts 
with 25 administrators and 2^8 teachers. 
The accompanying Instructor's Manual 
was field tested in the' same 18 school 
districts con^currently wi^th the Pro- 
grammed Course , Results from ,the 
field testing indicated wide-spread 
acceptance and Usability of/the two, - 
programs in aU implementing sites. 



as^ 

have 



Program-Activities have been 
Initiated in some 1800 districts 
of September 197^. Participants 
expressed satisfaction with the results 
and have indicated program usability in 
educat i onal plannjng. 



Costs 



Kit, 



One "Goal Development 
costi.ng $60, contaihs 
display boards 
cards of 18 goals 
sets of discs ^ 
envelopes 

sets of goal sheets, 
sets of directions, summary 
sheets , tal ly sheets 
160 sets of perceived needs 

i nstsruments . 
12 A,dmi nistrator's Manuals : 
Educational Goals and Objectives 
6 workbooks, A --Prog rammed Course 
for the Writing of Performance 



60 
60 
60 
60 
60 
60 
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Objectives 

Replacement materials may be 
puechased either separately or in a 
$28 kit containing 
• 60 cards of 18 goals 

60 sets of goal sheets 

180 perceived needs instruments 

15 sets of di rect ions 

2 summary sheets 

•Order forms listing costs for 
separate materials are included in 
the ''Goal Development Kit." This 
ki t "inclqdes six programmed work- 
books and one instructor's manual. 
Additional training materials for 
participants and instructors can be 
ordered separately for $1.00 apiece. 

Lists of consultants available 
throughout the United States can be 
obtained from the professors named 
below. Cost varies. Dr. Rose 
charges only travel and per diem. 

Dr. Carroll L. Lang 
California State University 
Northrldge, California 9132^» 
(218) 885-2591 ^ ' 

Dr. B. Keith Rose, Director- / 
. Northern Cal i fornia' Program 
Deve lopment. Center 
California State University 
Chico, California 95926 
(916) 3A2-I838 



Comment 



Educational Goals and Obiectives 
is a total package, complete and 
self-sufficient. It contains aH the 
materials necessary to conduct the 
program, from select i^on of community 
representatives wfio rank the dis- 
trict's goals, to the actual w^^'ting 
of program level objectives to match 
the goals : 

t 

It provides a range ^f alterna- 
tive procedures for selecting the 
community group. The advantages and 
disadvantages of each ^a 1 ternat i ve " 



are discussed, including h< 
alternative is 1 ikely to bi 
by the community, how eact 
the policy-making pow^r 
intendent and school boafd 



each 
rece i ved 
may affect 
the super- 
and how 



difficult each is to ef/ect. 

It appears that t/he program would 
be easy to adopt in variety of 
school districts and/that little 
change would be neecled to put it to 
use. The develope/s recommend that 
for best results, A^ew changes be made 
and that the procedures outlined be 
adhered to as clqfsely as possible. 

The developfers make the following 
statement about /the ;bes t use of their 
program: 



The success 
conversely, i ts 
pends upon severa 



o^ this program (and, 
I aCk of success) de- 
factors : 



2, 



It is abs61utely essential 
that administrators play an , 
active role in securing the 
services of the community 
participants . 
It is imperative that the 
members of the .various task 
forces be competent in their 
understanding of, aqd ability 
to write, performance objec- 
tives.* 

The management procedures 
outl ined in this booklet 
should be followed as closefy 
as local conditions and 
educational* pol icy permit. 
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DETERMINING INSTRUCTIONAL PURPOSES: 



A 

Training 
Package 



Target Aud4ence 

This program is for assistant 
superintendents, curriculum directors 
principals, department heads and 
lead teachers involved in long-range 
planning and decision making at 
grades kH2. It is also appropriate 
for graduate students training for 
^administrative credentials and 
positions. 

Participants need not be pro- 
ficient in instructional planning and 
management, but must be familiar with 
the operation of schools. 



Content Focus Planning .and 
selecting instructional ' 
objectives. ^ ^ 

Type of Product Materials foT^ 
conducting three, diffel-ent 
two- to three-day workshops or 
for use in a regularly scheduled 
course. 

Availabilit i f isy/*. 

Deve toper Far West Laboratory for 

Educational Research and 
Deve lopment 
1855 Folsom Street 
San Francisco, California 
9^103 . ^ 



Distributor 



In Cal i fornia': 
Association of 
Cal ifornia School 
Administrators 
2212 Dupont Drive 
I rvine, Cal ifornia 
9266i» 

Elsewhere: 
Contact developer. 



Goals and Objectives 

The materi^als are org^ized into 
' ^hree units, each addresslfa itself 
to a specific task: setti^ goals, 
analyzing problems, and deriving* 
objectives. The comp let ion of a 
workshop in any one of the units, or 
of the three taken in sequence, would 
result in 

setting 3oals--imprnwi:>H know- 
ledge of^and attitude toward 
deriving district goals; also, 
refining goals that reflect the 
preferences and needs of stu- 
dents, parents, and ot+ier school 



30, 



'28 



ERIC 

hnimnmrrT-uaaiJ 



r 



related groups . 
analyzinci prob lems- ^sharpened 
ski I Is in Ident i fying instruc- 
tional problems and defining 
problems using existing and 
desired student outcomes; 
collecting information needed 
to analyze problems; and de- 
termining the validity and 
relative sei^iousness .of 
problems. 
deriving objectives- - rea 1 i z i ng 
the- i mportance of deriving 
object jves. as part of the goal- 
refinement process and ulti- 
mately translating goals into 
more specific statements of 
desired learning outcomes, 
i.e., into usable objectives. 

Materials and Equipment 

The materials for the workshop 
are contained irt* three separate units 
for trainees and a handbook for the 
worJ<shop coordinator. The units and 
the handbook inake up what the devel- 
opers call the Purpos i ng Package . 

The trainiftg material in each 
unit is organized in separate modules 
consist ing'of reading assignments, 
input materials, worksheets, and 
written feedback. The three units,' 
Setting Goals . Analyzing 'Problems , 
and Deriving Ob ject i ves , are printed 
on three-hole punched paper to fit 
^tandard size notebooks and are 
packed separately. Notebook covers 
are not provided. 

The Deriving Objactives unit has 
a separate booklet, a programmed text, 
that Is used In module three of the 
unit, "Analyzing Sets of Curricular 
Objective^.'* 

A Coordinator's Handbook is 
printed on three-hole punched paper 
and completes the materials that make 

the Purposing Package . The hand- 
book contains a summary of the organ- 
ization and content of each unit and 
can be used in the workshop with any 
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one of them. 

How to Use 

The Purposing- Package is designed 
so that participants function on their 
own, either individually or in groups, 
much of the time. A coordinator who 
acts as an organizer, guide', and mon- 
itor is essential if the training units 
are to achieve their intended results. 
This person, who should be an adminis- 
trator in a school or district, or a 
professor for a university-based course 
also has the^option of selecting a 
t rai ni ng uni t that is appropriate for 
the immediate or prospective job re- » 
. sponsibi 1 ities of the participants, 
and, if more than one unit is to be 
used, he or she should determine the 
sequence of presentation. 

The units are similar, but not 
identical. In organization and types 
of materials. They are suitable for 
an inservice workshop, or preservice 
training at the university level, for 
any number of groups of from three to 
six members,' either as part of a reg- 
*^ ularly scheduled course or a special 
class or workshop. 

The. Cocfrdi nator 's Handbook is a 
comprehensive and informative tool. 
It provides ffrie coordinator with the 
necessary instructions for conduc'ting 
the workshop , information on the 
development of the training units, and 
detailed instructions about each unit's 
activities. Setting Goals is divided 
into four modules, each "coveri ng a 
specific skill necessary for formulat- 
, ing a district goal statement. Analyz- 
i ng Problems has s Ix modules deal 1 ng » 
with how to^determine which of the many 
prob 1 ems that i ns t ruct i ona 1 p 1 anne rs 
face most deserves thej r attention,. 
Deriving Objectives contains four mod- 
ules and focuses on how to translate 
goals into usable objectives. 

A minimum of 10 to 15 hours is 
required for completion of each C^nit, 
with an^estimated time of two to^'three 



hours per nx>dule. The total time * 
varies with particular groups depend-^> ' 
ing on their motivation, level of 
♦skills, and how thoroughly they want 
^ , to cover the iflaterial.^ Workshops 
can be either in small segments 
lasting from one ta three hours or 
concentrated into a two- or three- 
day session. The Coolrdi nator ' s . 
Handbook offers suggisstions for 
alternate scheduling of'the program. 

' , Trainee Assessment 

7 . ■ ■ 

Immediate feedback is provided 

l-n the self-test exercises by com- 
paring the participant's responses 
with the suggested responses. Since 
worksh*eets .are supposed to reflect 
team G<5nsensus, the work dong by 
each individual is not considered 
complete until another^ team member - 
has reviewed it. At the er»d of each 
module, teams exchange comments on 
the train^ing activities and discuss 
problems they encountered. Effec*^ 
tiveness of the trai.ning session is 
revealed when acquired skills can be 
Immediately t rans I at^d 1 nto rea I - I i fe 
situations. 

The eval uation of overal I per- 
formance of 1 nd'ividuals , for purposes 
7 of grading or rating, depends on ,the 

setting of the sessions. The devel- 
opers suggest giving additional 
assignments individually, so that the 
participants can use the process 
learned in real-life school- s i tuations 
of their own choosing and write a pa- 
per-^ describing the. process used. 
Enough time should be al Towed for the 
participant to absorb t'he skills' cov- 
ered in the unit(s) before the 
assigninent is due. 

Product DeveTopment and Evaluation 

The three^nits of the Purposing 
Package were developed concurrently, 
but field tested Independently at 
different sites. Setting Goals was 
field tested at three school districts 
as an Inservice workshop, and at two 
universities. Between October 1971 
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and Febr^uary 1972, a total of 7^ Persons 
participated. 

Analyzing problems was field 
tested at a university on four differ- 
ent occasions with participants 
from March to July, 1971. Pre- and 
posttests were given to evaluate 
trainee achievement of skill objectives. 
Tests were Identical in content and 
required between^two and three hours 
to complete. Results were judged by 
independently trained raters who were 
unaware of the field test design. 

Deriving Objectives was field 
tested in the same cycle as the other two 
units. The main field test was conducted 
at five s i tes ^betweoi'ij^tober 1971 and 
March 1972; 12 to l68^pfersons from 
graduate schools of education, school 
district inservice programs, and pro- 
fessional organization workshops 
participated at e^ch site. The coor- 
dinators for the field tests were the Far 
Vtest Laboratory staff and on-site pro-^ 
fiBSSionals. Affective data were 
gathered to determine the acceptance 
and value of the training unit. The 
analysis of the findings showed it had 
more than satisfied the affective 
requirements for its target audience. 
« 

Costs 

Unit I, Setting Goals $8.95 
Unit 2, Analyzing Problems $8.95 
Unit 3, Deri ving Objectives $8.95 
'"Sample Kit" (sample pages 
of each uni t) ^ $| .00 

A check or money order must accom- 
pany each order.. California residents 
should prepay applicable sales tax. 

Comments x 

Each unit is a self-contained 
training tool designed to Incorporate 
into the course or workshop as many 
facets of an actual school setting as 
possible. The Settij j ig Goals unit 
demonstrates how establishing prior- 
ities lays the foundation for sound 
decisions about al locati ng time , money ,^ 



and energy. The simulation materials 
provide a common working base for 
participants from different school 
backgrounds, and some of the exer- 
cises project anticipated community 
Input. 

From Analy2:ing|,,Problems the 
participants learn long-range plan- 
ning techniques, but the'Qnit Is not 
designed to deal with crisis or 
confrontation situations. The ad- 
vantages of the approach include 
(1) effecti ve use of resources by" 
treating problems in order of 
priority; (2) the establishment of 
realistic goals and Objectives from 
which student outcomes become clearer; 
and (3) communication to -the community 
of the schooTs order of priorities.^ 

The flex of Oerlvinq 

Objectives encourages participants 
to approach exercises critically and 
to challenge the developer's suggested 
responses^ The time needed to com- 
plete the program Is determined by 
participants -and coordinator and 
depends on how intensivjely they 
plan to study the materials. 
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INTERPERSONAL COMMUNICATIONS 



Content Foaua^ Interpersonal 
1., communlcat-fon skills; group and 
brganlzationdl condl tlons affecting 
commun I cat ion , 

Ti^po of ProduQt Inservlce .work- 
shop training materials emphasizing 
group^ activities. 



Availabilify Since October 
Five-year copyright expires 
October 1977. 



1972. 



Develover Charles Jung and others 
Northwest Regional 

Educat lona 1 Laboratory 
L Indsay Bu I Idl ng 
; • ^ 710 •S.W, Second Ave. 
. ' : , Portland, Oreigon 
97204 



Distributor 



X I com, I nc. 

RFD 1 , Sjterl ing Forest 
Tuxedo, New York 
10987 
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Target Audience 

The workshop materials were 
designed primarily for K-12 teachers 
in all subject areas but can also i>e 
used by principals, superintendents, 
assistant superintendents, counsel- 
ors, curriculum specialists, and 
paraprofessIonals--in fact, by any 
school group striving for Improved 
communication. Other groups who 
have used the product are college, 
professors, state education depart- 
ment personnel, and high ^chool 
students. The materials are espe- 
cially useful for staff members 
engaged In coMabora t ive act i vl 1 1 es 
such as group planning, team 
teaching, modular scheduling, and 
differentiated staffing. No 
prevlous^tra [ning Is required of 
participants. 
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Goals and Objectives 

The workshop is intended to ^ 
increase the participants* knowledge 
of communication processes and of 
varlous'^groups and organizational / 
conditions affecting communication. 
It Is also Intended to help partic- 
ipants recognize and deal wtth ' 
these processes and conditions in 
their own school. In addition, the 
workshop should heighten each, 
participant's awareness of his or 
her own personal style of communi- 
cating and should Improve communi- 
cation skTI Is through practice in 
group actiyi ties. 

Materials and Equipment 

The training program consists 
of 20 Instructional units made up 



of sets of handout sheets, taking 
approximately 90 minutes a unit to . 
complete. Films accompanying some 
of the units either demonstrate the 
skills being (earned in that unit 
or illustrate a situation where they 
are rteeded but absent. With the 
' exception of the introduction and 
test units, each unit concentrates 
on spegi f ic concepts . 

Unit 1 - "Introduction" provides 
a survey of the entire program and a 
firm showing three teachers at -cross- 
purposes. After watching the film, 
participants discuss the kinds of 
jii«iiT]^ni cat ion problems they observed. 

Unit 2 - "Paraphrasing." In this 
unit participants practice orally 
paraphrasing spontaneous statements 
, made by others on the team. Para- 
phrasing helps to clarify one*s^ 
understanding of what others are 
trying to communicate. 

Unit 3 - "Behavior Description." 
Participants describe one another's 
actions as specifically as possible 
without making assumptions about 
the other person's motives, attitudes, 
or .attributes . 

Unit ^ - "Description of Feelings 
and Perception Check." The unit pre- 
sents criteria for describing one's 
own feelings accurately and verifying 
one's perceptions of others' feelings. 
It Involves analyzing statements, 
completing written exercises , and 
discussing given examples. 

Unit 5 - "Nonverbal Communi cat ipn 
and Perception Check." Participants' 
learn to recognize and interpret 
nonverbal cues, such as gestures and 
facial expressions. They discuss 
the'effects of their own .nonverbal 
cues. They observe nonverbal behav- 
ior, report thei r observations 
to each other, and becpme more 
familiar with their own responses 
by recalling how they reaqted in 
particular situations. 
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Unit 6 - "Trte Concept of Feedback' 
details techniques for sharing reac- 
tions to each other's behavior, par- 
ticipants request that their team mem- 
bers react-to specific behaviors; each 
person attempts to use the techniques 
fo-r sharing reactions set forth in 
the unit. 

Unit 7 - "Expectations and 
Communication" cites the effect 
expectations have on how one inter- 
prets messages and considers phenomena 
such as the "self-fulfilling prophecy." 
In one of the exercises, half the 
group is given a favorable descrip- 
tion of a character they are abou-t to 
see in a film, and the other half is 
given an'entirely different, unfavor- 
able description of him. The exercise 
shows how a preconception of a char- 
acter can Influence one's view of 
his actions. 

Unit 8 - "The Interpersonal Gap" 
discusses the differences between 
one's Intentions In communicating and 
the actual effect produced. Exa^les 
are given of situations In which in- 
tentions and effects are highly dis- 
parate. Participants describe how 
they would react In given situations 
and why. 

« 

Unit 9 - "The Effects of Feelings.' 
The Importance of recognizing and deal- 
ing with emotions honestly Is the 
subject of this unit. After watch I ng 
a fllgi In which anger, admiration, and 
helplessness are expressed , pal rs of 
team members discuss how they would 
have behaved In each role and suggest 
alternative ways for communicating the 
same enptlons. 

Unit 10 - "Matching Behavior with 
Intentions" examines the problem of ' 
achieving qqreement between Intentions 
and behavIoK In this unit, partici- 
pants Identify their own Intelitlons 
toward other team members and consfder 
how they might convey these Intentions. 

Unit 11 - "Open Communication: 



Freeing and Binding Responses.'* 
These materials explore the condi- 
tions that foster openness, trust, 
and good communication and those 
contentions that foster suspicion, 
mistrust, and loss of autonomy. 

Unit 12 "Communicating about 
Interpersonal Relationships." This, 
unit explains the "Circular Process 
Model," a construct representing 
psychological variables influencing 
^ communication. Variables include 
.self-if^age , preconceptions about 
the other person, personal tenden- 
cies to certain types of behavior, 
perception and interpretation of 
other people's behavior. 

• Unit 13 - "Roles and Patterns 

of I nterpersonal Communication," , 
These materials call attention to 
the inflcie-nce of roles on how in- 
formation is sought, conveyed,, and 
received. Part i ci pants . ana lyze the 
patterns of communication that have 
developed in thel r teams. 

Unit U - "Norms and Communica- 
tion" concerns the adaptation of 
behavior to a situation. Partici- 
pant's take note of^Jibe frequency 
and types of exchanges within their 
group. 

Unit 15 - "One-and-Two-Way 
Cammunlcatlon" defines "direction" 
In communication: "In a one-way 
process, A communicates with B... 
In a two-way process, A communi- 
cates with B; and B, In turn, also 
has the opportunity to communicate 
with A." In the exercise, one 
member of the team describes two 
geometric figures to his or her 
teammates without displaying the 
figures or using hand gesti|T]^s. In 
the first trial, the listeners may 
not ask questions; in the second, 
they may. The accuracy of guesses 
about the figures and levels of 
frustration are recorded. 

Unit 16 - "Communication 



.Patterns in^ the School BuITdlng."' ThI 
deals with formal and ^Informal pat- 
terns of communication and the wa^s 
In which different peoplp aid or 
hinder communication in a given situa- 
tion. Participants analyze communi- 
cation patterns in- their own school. 

Unit 17 - "Communicating Under 
Pressure." The unit consists of an 
exercise in which teams must solve 
a problem demanding cooperation. 
Each team member has some of the 
Information needed ^o solve the 
problem, but no one person has 
enough to solve the problem alone. 

Unit 18 - "Assessment of Know- 
ledge." Partlclpants'assess their 
progress In the program \r\ two ways. 
First they take a content test on 
the preceding units. Second, ' 
participants rate themselves and . 
are rated by Jtheir teammates on 
thelr'skllls. 

Unit 19 - "Improving My Skills." 
Participants design a personal ' * 
learning situation with three 
elements: the skj 1 1 selected for 
Improvement, the situation In which 
It will be practiced, and the means 
of assessments 

Unit 20 - "Developing Support 
for Continuous Learning." Head- 
ings In this unit are "Setting 
Goals for Improvement" and "Devel- 
oping Support for Continuous 
Learning." The latter suggests 
how participants may develop 
helping relationships In which th^y 
may continue to Improve their 
skills. 

Appendices containing the 
scripts of the films and audiotapes 
fol low the unl ts . 

How to Use 

. The 20 units of the program 
disarranged sequentially and build 
upoVeach other. Fon this reason. 
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It is not recorwDended that units be 
-skipped, taken out of order, or that 
the workshop'be condensed. Parti ci- ^ 
pants should be present for all' 
sessions. Most exercises are team 
endeavors., and full team participa- 
tion Is essential for their success. 

Best results are obtained when 
entire school staffs participate in 
the workshop. Gains in rapport and 
cooperation and improved communica- 
tion can thus be carried over to the 
school setting. Conflict and ob- 
struction by nonparticipating staffs 
can be avoided. 

Whenever possible, the 20 units 
should be covered in five consecutive, 
days, four units each day. Other 
arrangements might be two sessions 
held one week apart, each lasting 
two and one-half days;, a one-day 
session per v;eek for five weeks; or 
daily meetings aft^r school until 
the units are completed. The time 
schedule should be agreed upon l^y 
all concerned prior to the workshop. 

The number of participants can 
be as few as 12 or as many as 36, with 
2k the recommended size. In any case, 
the total* number should be a multiple 
of six. Exercises require six mem- 
bers to each team. Fewer than six is 
unworkable and more is unwieldy. 
Whenever the number of participants 
cannot be divided into groups of six, 
however, it is better to have seven 
or eight on a team than to have less 
than six. 
\^ 

The leadeKsh^uld be someone who 
, has already participated in the work- 
shop iit least oilce and thoroughly 
reviewed the material in each unit. 
Further training or expertise is not 
necessary. The leader serves more as 
a coordinator and guide than as an 
instructor. Beginning leaders should 
work in teams of tw^ with no more than • 
12 participants. in larger groups, 
the novice leader should work w|,th an 
experienced leader. 
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Workshop activities include 
reading printed materials, completing 
wri^'tten exercises, watching and dis- 
cussing films about school- interactions 
listening to an audiotape, and engaging 
in numerous group exercises. These 
exercises are the core of the workshop. 
Through the exercises, participants 
experience and resolve communication 
difficulties, increase perception of 
their own and others' commun I cat i^on 
styles, and develop communication 
skills and techniques. Exercises In- 
clude paraphrasing what another partici- 
pant has said until he or she agrees 
that the message has been understood; 
describing other people's verbal and 
nonverbal behavior nonj udgmental ly ; and 
activities such as describing diagrams 
without displaying the diagram 
using gestures. This last exercise 
illustrates the Importance of non- 
verbal expression in total communlcatron 

Trainee Assessment 

A brief pretest precedes use of the 
materials, and short fill-in and multi- 
ple choice self-tests accompany each ' 
unit. Unljt 18 Is the posttest for all 
the preceding units. All tes t-, I ns t ru- 
men ty serve as a-quick review and sum- 
ma^l^n. Answer keys are provided. 

Major means of evaluation are 
assessing one*s own effectiveness In 
the practice sessions and receiving 
feedback from other participants. 

Product Developnient and Evaluation 

Interpersonal Communications was 
developed at Northwest Regional Educa- 
tional Laboratory over a thrjee-year 
period. Dr. John Wallen wr^ote the in- 
itial materials, based on "I nformat I on 
from the National Training Laboratory 
Institute for AppI led- Behavioral 
Sciences. The training design draws on 
the work of" the Center for Research-on 
the Utilization of Scientific Knowledge, 
University of Michigan. Materials "and 
training procedures were Integrated Into 
an Instructional system by a three-. 
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member team at Northwest Laboratory 
and, In cooperation withXIcom, Inc., 
audiovisual materials were produced. 
After , a series of field trials and a 
major field test, the materials were 
revised. 

Results of seven field tests 
with 167 participants In various 
parts of the country Indicate the 
following rates of achievement: 

Recognize descriptions of 

behavIor--89^ 
Paraphrase--90^ 
V* Recognize a perception check--8^^^ 
Discriminate among descriptions 

Qf ^behavior, descriptions of 

feeling, and expressions of 

feeling— 
Know appropriate guidelines, 

for giving feedbacl<--7^^ 

and recel vi ng feedback--8'^^ 
Recognize norms^-gSJ^ - 
Discriminate among paraphrase, 

f-eedback, and perception check 

"in ^ 

Demonstrate a rudimentary knowl- 
edge of the circular process, 
factors affecting open and 
closed communication, the 
consequences of one-way and 

\^wo-way communication, and the 
"DoTLook-Lear^" model of 
learning--75^ 

Affective responses include: 

Offering new insights, new ways 

of viewing old problems — 87^ 
Spoke to important issues, 

vital concerns--78^ 
Gained new insights about my^ 
styj.e of communicating — 85^ 
Ideas , .ski 1 Is , methods can be 
used l)nmed lately- -89% 
Provj^d6d real "how to" help 
my actual work — 85^ 
Ian to use the ide^s, skills 
and/or^-materials presented in 
this workshop as an integral 
part of your work — 9^^ 



Comment • 

The possible benefits of the 
instructional program are far- 
reaching. These techniques show 
promise for improved group prob 1 em , 
solving and crisis Intervention. 
Instructional programs can take on 
new 1 I fe when Innovations are shared 
and new IdeaT and solutions are 
generated by group energy. Collab- 
orative endeavors such as team 
teaching and di f ferentiqjted staffing 
can become more effective. Communi- 
cation with the school boai-dr parents 
artd the community as a whole can be 
upgraded. Finally, enhanced commu- 
nication in the classroom--better 
Instruction and greater^ unders tand I ng 
between studen't and teacher--is the 
ul timate benef I t. ' • 
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Costs 

Requi red 
I terns 



Quant i ty 
Needed 



Cost 
Per| [tern 



Leader's Manual -vsee below 

Participant materials One set per participant 

N'ine 16mm black and One set 

whi te sound f i 1ms ; 
one audiotape 



$15.00 

$1 KOO/set 

$175.00/set 



Replacement 
Rate 



Reusab le 
Consumab le 
Reusable 



'^9? one experienced, leader for groups of from 6 to 36 members 

{lU members recommended size). Novi ce .leaders should not be assigned • 
more than 12 participants unless working with an experienced coleader. 

If training is to be conducted during the. school year\ i t may be necessary 
participant) re leased-t ime and pay for substitutes (30 hours per 
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PRENTICE-HALL TEACHER dOMPETENCY DEVELOPMENT 
SYSTEM ^ ^" 



Cont^ht Foam Staff development in 
cu r r ( cuj urn , i ns t rue 1 1 on , and evalu- 
ation ski lis. 

Ti^po Product^ Twenty-^sTx self- 
Ir»structlonal booklets 

> ' Avctilabilitff Since 19/3. 

Dovelopev W. L. Popham ^ 
^ ^ Eva L. Baker 
'Graduate Schopl of 
Education, UCLA 

DiQtHbiitov Prent Ice-Hal 1 , lhc. 

Englewoojl 'CI I f fs , 
New Jersey 
07632 



Target Audience 

The product has boon used for 
Inservice sta/f development at J:he 
elqmentary, secondary, and college 
levels. It hos also been used as 
part of preservic^ teacher training 

unl vers! ties 'and colleges. It 
has not been used in trai.ning admin- 
istrators but could be part uf an 
iriservice program for research and ^ 
development personnel who write 
instructional materials. 

Goals and Objectives 

The goal^j of the system are 
tjireefold: to assess th<^ teacher's 
present competences and diagnose 
areas where development is needed; 
to set goals for these specific areas 
and supply materials to fulfill them; 
and to evaluate the achievement of 
individual goals and general program 
goals. 

' Each of the 26 se I f- 1 ns t ruet lona 1 
booklets has Its own set of behavior- 
al objectives. 



Materials and Equipment 

The primary instructional material 
of^the Prentlce-Hal 1 Teacher Competency 
Development System is contained In 26 ^ 
sel f-f nstructlonal booklets. The com- 
ponents ^f the product are as follo/is: 

The Administrator's Guidebook de- 
scribes the system, various approaches 
for installing s?nd using the programmed 
materials and testing Instruments, and 
additional resources feo complement the 
system. The appendices contain descrip- 
tions and objectives for each of the 26 
self-instructional booklets, two self- 
reporting measures to evaluate a teacher 
training program, and the scoring keys 
for t?he testing materials. 

The 26 self-instructional booklets 
are. divided Into three main content areas: 
curriculum, instruction, and evaluation. 
.Each programmed booklet contains behav- 
ioral objectives, p^ractlce exercises, a 
mastery test, and answers. These booklets 
are complete In themselves and can be used 
In any arrangement. Below are the titles" 
and a descriptive statemenjK^ about each 
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booklet. 
Curriculum ' 

1 . Educati onal Object ives 
enables the reader to distinguish 
between behavioral and nonbehavioral 
objective's and to revise npnbehav- 
ioral objectives into performance 
teVns . ^ ' 

2. Selecting Appropriate 
Educattonal Objectives helps the 
reader distinguish between three 
kinds of pupil behavior and develop 
a positive' atfitude toward behavior- 

• ally stated objectives. 

3. A' Curri cu lum Rat ionale 
explains a plan for developing 
6bjectives. 

Deciding on Defensible Goals 
Via Educational Needs Assessment 
acquaints the reader wrtfx a specific 
procedure for determln-ing educational 
. goals . ^ 

5. Humanizing Educational 
Objectives descbrfSeg wayc fr^ gono rale 
and implement nonmechanica 1 , "human is 
tic** objectives . 

6. IdentifyJng Affective 
Objectives describes a strategy 

• which identifies measurable affective 
objectives,' ^ 

7. Defining Gontent for Objec^ 
ves describes and promotes the 

Idea of content generality in objec- 
tives. X 

8- Establishing Performance 
Standards Identifies specific achieve 
ment expectations to be Included in 
objecti ves • 

Instruction 

9- AnalyzVn^and Sequencing 
Learner Behavior provides practice 
in analyzing objectives and formula- 
ting entry and en route behaviors, 

10- Appropriate Practice enables 
, the reader to distinguish between 
different ^kinds of pupil ^tivities 
and write out learning activi^iies 
that match tffem. 

1 1 • Knowledge of Results 
describes the type of practice 
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-situations in which the learner has ' 
an opportunity to deterjriine wfiether 
he or she is right or wrong.. 

.'2. Perceived Purpose describes 
four techniques t«iat can be used to 
/promote suitable learning activities. 

13. Instructional Tactics for 
Affective Goals emphasizes the impor- 
^ tance of affective goals and describes 
techniques to achieve them. 

Wrhten Plans fof Classroom 
Instruction describes two types ofo 
written instructional plans: the 
teaching unit and the lesson plan. 

'5. Individualizing Instruction 
distinguishes between individualized 
objectives and i ndi v i dual i zed instruc- 
tional procedures and describes three 
short organizational patterns needed 
in indiv/r diiaj ized instruction/^ 

16. Discipline in the Classroom 
offers six rules to guide teachers 
in maintaining classroom discipl ine. 

'7- Opening Classroom Structure 
describes characteristics of the open . 
classroom and suggests ways to imple- 
ment an operf structure in particular 
si tuations . jT 

'8« The Teaching of Reading dis- 
cusses an empirical approach to reading, 
describes objectij/es and instructional 
activities illustrating this approach, 
. ^n^^^Wdes practice in writing test 
flHlUlyQDPli^^sure reading skills; 

Evaluation 

'9- Evaluation discusses evaluation 
of^student achievement,- - . 

Current Conceptions of Educa- 
tional Eyalu^fron fami 1 i ;ir i ^oc the 
reader with some of the contrasting' 
theories of contemporary educational 
evaluators ^ 

21 . AJternative Wvenues to 
Educational Accountabiol J ty des c r i hes 
three educational accountability 
strategies: personal, professional, 
and publ ic. \ 

22. Modern Measurement Methods 
describes criterion- and norm-refer- 
enced testing techniques and shows the 
reader how tp decide, which is more 

te in a given situation. \^ 



appropriate 
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23. Writing Tests Which Measure 
Objectives provides procedures useful 
\p generating test items that meas^ur^ 
objectives. 

2k. Constructing Teaching 
Performance Tests des'crlb^s the 
rationale and character i st i cs" of 
effective teaching performance tests. 

25- Using Teaching Performance 
"^ests describes 'the role of and 
promotes the use of teaching perfor- 
mance tests. 

2$. I nstructional Supervision : 
A Criterion-Referenced Strategy 
describes the role and activities" 
of an instructional supervisor who 
uses a criterion-referenced strategy. 

Diagnostic Test contains test 
items rislating to each of the self- 
instruct ional'booklets . It is used 
.to help a teacher determine the areas- 
of instruction he or she should 
develop. " 
A Comprehensive Mastery Test 
all of the competences taught 
booklets. It IS used at the' 
the instructional program to 



covers 
i^n the 
end of 
assess 



teacher's attainment 



ob ject i ves . 

A Personal Profile Sheet provides 
the learner with an incfividual progress 
record of hfs or her performance on 
w;j;he^ Diagnostic and Mastery tests. An 
''^bp'fMonal resource that complements the 
system is a textbook, Systematj.c ^ 
I nstruction , written by the developers 
of the system, Popham and Baker. 
Afmed at teachers, this text provides 
the fundamentals of instructional 
decision making. 

There are separate audiovisual 
materials which can be coordinated 
with the product or used separately. 
These materials consist of 30 
f i Imstri p-tape programs that cover 
the same content found in the booklets 
plus four other topics: 

"^-"Teacher Aides in the Classroom" 
prepares teachers and aides in the use 
of paraprofesst onals in the classroom; 



"Systematic Instruct ionaj Decision 
Making" provide^ an overview of a 
ccm^equence-based instructional mode J ; 

^^"Alternative Measurement Tactics 
for E^di/cational Evaluation" describes ' 
a four^category system to use in 
generating measurement plans; * 

"Experimental Designs for School 
Research" describes formative and 
summatjve evaluation designs. 

Each of the 30 programs includes 
color illustrated filmstrips of 30-^P 
frames, accompanying audiotape nar- 
ration of approximately 30 minutes^ an 
instructor '^s manual which suggests 
technplques 'for use, statement of ob- 
jectives, reports of vaUdation 
studies on program effectiveness, a 
copy of an optimal response sheet to 
b^.used as the program is viewed, and 
a copy of the pre- and/or posttest 
measuring the. program' s objectives. 

How to" Use 

The instructional materials (26 
booklets) and the testing materials 
(Diagnostic Test, Mastery Test) can be 
used with or without -an administrator 
or coordinator. If a coordinator sets 
up and administers the training program, 
the Administrator's Guidebook suggests * 
a variety of approaches for managing 
and administering the instructional 
and testing materials. If there is no 
coordinator, the Guidebook can be con- 
sulted by an individual teacher for the 
following information- suggested ways 
to use the booklets, names of additional 
resources , and. scoring the Diagnostic 
' and Maste^r/ Tests , 

The instructional booklets can 
be used separately or i,n any combin- 
ation. Each booklet module is ' 
programmed, I5 to4|^ pages long, 
and requires abo^t 30 to 60 minutes 
to complete. The booklets can be used " 
in an unstructured or highly structured 
program, depending upon^ the n^ds of 
the users. One%equence in which th? 
modules could be used is according to 
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the three main content areas (see 
Materials and Equi pment) . 

A sequence that could be 
used to teach affective objectives 
is 

^. Deciding on Defensible 
Goals Via Educational Needs Assess- 
ment ; 

5» Humanizing Educati onal 
Ob jecti vesT ] ' 

Tdentifying Affective 
Ob ject i ves ; , - . 

1 3 * Instructional Tactics ^for 
Affective Goals ; and 

17* Opening Classroom Structure . 

Modules which have relevance 
to teaching reading sITills are 

1. Educational Objectives ; ^ 
9. Analyzing and Sequencing 
Learner Behavior ; I 

10. Appropriate Practice ; „ 

11. Knowledge of Results ; 
18. The Teaching of Reading ; 
22; r Modern fjpasurement Methods » 

and 

23 . Writing Tests'^ Which Measure . 
Outcomes . " - \ 

Trainee Assessment 

The, Diagnostic Test is a short 
test taken at the beginning of the 
program. The items are divided into 
content sections that correspond to 
the i/iformatlon in the booklets. -The 
results of this test indicate what 
booklets should be read to improve 
iieeded skills. , Each booklet contains 
a mastery test and answers . .When a 
teacher finishes all the booklets, 
he or she^ takes the Compre^hensive 
Mastery Test» which i s longer .than 
the ftrst test and covers all the 
competences taught in the 26 booklets. 
If the teacher uses only *ome of the 
booklets, he dr 'she can select the 
items in the Mastery Test that relate 
to those booklets. A Personal Profile 
Sheet is provided for each user to* 
keep a record of progress on the 
Diagnostic .md Mastery tests. The 



scoring*. keys to these two tests are 
in the Administrator's Guidebook . In 
addition to the'se testing techniques, 
there are two brief self-repbrt 
me'asures tlh3t may be used to evaluate 
th^success of a- preserve ce or in-, 
service teacher training program. 

Product Development and Evaluation 

Each of the modules was developed 
independently arfd empi ri cal ly over" 
a period of ten years and later incor- 
porated into a total system. Field 
tests were conducted wi th i nservice 
and preservice teachers in groups 
ranging 'from 15 to 150 members. An 
outside consultant firm conducted a 
marketing analysis survey for Prentice- 
Hall to determine usfer reaction to v. 
the whole system. ^A random sample of 
purchasers were interviewed by tele- 
phone. Highly favorabl'e reviews were 
received from school systems using v 
the system for inservice programs. 
There were isolateid negative responses ' 
from college, teacher-education programs 
complaining that the system was repe- 
titive and simplistic. Prentice-Hal 1 > 
IS negotiating to have field testing 
done on the total Teacher Competency 
System . ^ ~ I 

Costs ' • 

The system can be purchased in a 
package called a Starter Set for $199-95. 
Thi s set includes-*he Admi nistrator 's 
Guidebook , 12 copies of each of the 26 
self-instructional booklets, and 12 
copies each of the Diagnostic and Mastery 
Tests and Personal Profile Sheet in an 
open display box. copy of Systematic 
I nstructioh is also included in the 
package. ' The minimum purcljase number of 
booklets and testing materials is 12. 
Separate, pri ces for each component of 
the package are 
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12 copies of a boJyklet 
Administrator Guidebook 

12 copies of Testing Materia-ls (Diagnostic 
• and Mastery Tests, P^sonal Prof i le Sheet 



■ The f rims trip-tape programs are 
sold by^the developers of the product 
and. not by the distributor. Each 
filmstrip-tape program costs $I8.Q0. 
A" discount of 5% is given on orders 
of 10 or more. These programs can 
be obtained from Vimcet Associates, / 
Inc., P- 0. Box -2^71 A, . Los Angeles, 
California 9002^. 

Comment * 

— ■ » 
School administrators should 
know about this system for possible 
use as part of inservfce or pre- 
service teacher training. The system 
seems to have effectively helped 
teachers increase their skills. 
Results of the user survey showed that 
teachers responcted favorably to the 
sys'tem and felt it increased their 
teaching skills. The schools that 
have used it have been able to adapt 
it to their particular teacher's 
needs. The programmed booklets are 
small, short, "easy to use, and do 
not seem to be as threatening as a 
textbook. The developers of the 
system have responded to a current 
educational need and have met this 
need with a certain measure of 
success . 



36 pieces total) A 
Systemati c instructic^ 
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RESEARCH UTILIZING PROBLEM SOLVING (RUPS) 



Target Audience 

The Research Utilizing Probleffl 
Solving—Classroom Version was 
designed-for school admi n i s t r.ators 
at all levels and classroom teachers 
of al I subjects and grades . The 
Administrator's Version is intended 
spieci f i cal ly for administrators. 

Goals and Objectives 

The purpose of the training is 
to provide teamwork skills, techniques 
for retrieving and applying informa" 
tion, and ski H In defining, analyz" 
ing and solving problems. The 
training develops the following 
competences : 

Applying four guideline 



Content Foaus Teamwork skills, data- 
gathering techniques, and problem- 
sol viag ski lis. 

Type of Product Materials for conduc- 
ting instruction as an inservice ' 
workshop or t*n i vers ity course. 

AvaLtability Copyright 1970, Noh:h- 
west Regional Educational Laboratory. 

Deve toper Charl§,s Jung and others 

Northwest Regional ^ 

Educat i or^al Lal^oratory 
^00 Lindsay Building 
7)0 S.W. Secoad Avenue 
Portland, Oregon 
9720^ < 

Distributor Commerci a I -Educational 
Distribjuting Services 
P.O. Box 3711 
Portland, Oregon 
97208 

criterfa for writing a problem 
statement ; 

Paraphrasing >\n Interpersonal 
commun i cati ons ; 
• Using the force field diagnostic 
technique; 

Selecting and creating instru- 
ments for data gathering;" 

Di agnosi ng teamwork re lation- 
ships; 

Spotting and analyzing major 

results of data collected; 
Identifying one's personal 

style of operational i zing 
* d^imen'^i ons of teamwork behaviors; 
Utilizing concepts and skills 

of giving and receiving feedback; 
Using criteria for deriving 

implications from research 

findings ; 
Brainstorming action a I ternatives 



ERIC 



5 



to meet implications derived 
from f i nd i ngs ; 

- Applying guidelines for plan- 

ning and itnplementing action . 
a^l ternat i ves ; 

- Identifying and evaluatfng 

small group dynamics; 

- Planning a back home project; ^ 

- Evaluating solution plans; arjisT 

- Conducting a back home RUPS 

project. 

Materials and Equipment 

There are two versions of the 
Research Utilizing Problem Solving * 
( RUPS) marterials : a Classrpom 
Vers1 on intended primarily for 
teachers , and an Administrator's 
Version designed especially tor 
school administrators. Both provide 
instruction in the problem-solving 
process and ''in teamwork skij 1 s . They 
differ in the sample probl^pms and 
simulations presented. The Adminis- 
tri'ton's Version presents ^ypical 
interpersonal and argani zatipna 1 
p^blems that interfere with the 
proper functioning of the school. 
The examples in th'e Classroom Version 
deal wi'th in-class interpersonal 
/elations and their effect on 
learning. 

* The exercises utilize a-running 
simulation in^^hiqh participants 
'^assist Mrv J&nes" in solving her 
problems. In the Admfnistrator's 
yersj_orr, Mrs. Jones is a' principal 
in the participant's school district. 
Having discovered that the partici- 
pant has* just returned from a work- 
shop on school problem solving, she 
approaches the part i ci pant for help 
In solving her difficulties with her 
faculty: Jack of commun i cat i on and 
mutual support among the teachers; 
In the Classroom Version , Mrs. Jones 
is a teacher whose students apparently 
don't want to learn. The simulation 
provides a context for developing and 
practicing teamwork skills and skifls 
required in the Research Utilizing 
Problem-Solving (RUPS) process. 



The instructional materials for 
the students are divided into 16 
subsets, each of which provides 
instruction, exercises, and a brief 
test. In the leader's manual, each 
. subset is preceded by an instruction- 
al strategy which .includes directions . 
to the leader on wh^<t . to do next, a. - * 
listing of materials needed^, tim'rng, 
directions for participants, state- 
ment of purpose and obj^ectives for 
the subset, and rational for the 
procedure . 

Accompanying materials are the 
simulation audiotapes and the booklets 
discussing the value, selection, and 
design of diagnostic i nstruments -- 
Diagnosing Professional Climate of 
Schools is used with the Administrator '.s 
Ve rs i on and Diagnosing Classroom. 
Learning Environments with the CI ass- 
room Vers J on . One of the audiotapes 
is keyed to each version. 

How to Use 

Continuity is important in setting 
up the workshop. It is best to cover 
the first 1^ subsets in a five-day 
workshop and have two three-hour follow 
up sessions approximately three and six 
weeks after the workshop. Another 
possibility is to plan two blocks of 
two and one-half days each, scheduled 
one week apart, in "which case the 
follow up sessions would stj 1 1 have to 
be *arranged. 

Group size is also important. It 
is advisable to limit the group> to 2^ 
participants. Thirty is the absolute 
maximum. The total group wi 1 1 be 
divided into six member teams and 
many exercises are designed for trios 
and sextets. Teams may number seven 
or eight when absolutely necessary. 

Training is most effective when 
all participants are volunteers, 
attend all sessions, and have a clear 
understanding of the content and 
purpose of the workshop. 
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'Leaders should have experienced 
the training before attempting to 
conduct a workshop. Prior to the 
workshop the leader should review 
the materials, but beyond this he or 
she needs no further preparation. 
The role involves guidance rather 
than instruction. Participants 
learn chiefly from each other, Pre- 
. ferred leader-participant ratio is 
1 to^m. "However, new leaders 
should work with no more thgn 12 
participants. 

One of the leader's duties is 
to prepare large newsprint sheets 
bearing the workshop agenda and 
important points that have been 
covered during the workshop as it 
progresses. The leader, alst) cJistri - 
butes the materials, clarifies 
instructions, operates the tape 
recorder, keeps time for exercises, 
and sometimes leads group discussions. 
Materials should be arranged in 
advance of the rneetings. Leaders 
prepare the charts used ahead of 
time. The room should be large 
enough to allow participants to 
meet In groups of three and six 
without interfering with each other's 
discussions. Chairs and tables must 
be moveable for frequent rearrangement. 

RUPS Process 

The use of the materials requires 
an understanding of the KUPS process . 
RUPS Is a five-step, problem-solving 
process emphasizing careful analysis 
and planning before taking action. 
The phases of the process are (l) 
identifying the problem; (2) diagnos- 
ing the problem situation; (3) for- 
mulating action alternatives; (k) 
planning for action; and (5) planning 
a community RUPS project. 

The process may" draw on two 
sources: scientific knowledge and 
knowledge of the particular educa- 
tional setting. The former Includes 
theory, research findings, and 
methodology; the latter. Information 



about priority needs, resources, and 
existing innovations . Data-gathering 
methods are presented in the unit on 
diagnosis but may be used at any 
point in the process. 

* 

Problem identification is the 
first step in the process. This 
requires specifying the essential 
elements of the problem situation. 
Participants analyze problems by 
using the guidelines provided. ° They 
'work on simulation problems, problems 
they* ve -experienced in the workshop, 
and problems they would like to solve 
In their school setting- 

I 

. The second step is diagnosis. 
Using a technique Renown as **force 
field analysis," the participants 
analyze the problem situation to 
determine what forces are working 
for and against a solution to the 
problem- If more Information about 
any of these forces Is needed, the 
participants can either turn to 
extant research or undertake new 
research- Different data-gathering 
techniques are briefly described In 
the participant materials and are 
dlscussed.at greater length In the 
booklets. Diagnosing Professional 
CI I-mate of Schools , used with the 
Administrators* Vers^lon and DIagnos In g 
Classroom Learning Environments , used 
with the Classroom Vers Ion . 

In one typical exercise, parti- ' 
cipants write a force field for 
Mrs. Jones* problem. After ranking 
the forces, participants select a 
force on which more information Is 
needed y determine possible sources 
of Information, and select or design 
two data-gathering techniques. Some 
of the participants Invent responses 
to the data-gathering instruments. 
Then the team considers the usefulness 
of the *'data*' collected, the efficacy 
of the data-gathering techniques, and 
whether there might be better ways 
to collect the desired Information. 
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\ The next step is the formulation 
of action alternati^ves . To do this, 
the participants first derive impli-'' 
cations about the particular situation 
from the data. These implications 
suggest what the objectives should 
be. The alternatives are the 
specific recommendations for achiev- 
ing the objectives. The group tries 
to think of as many alternatives as 
possible before settling op a plan 
of action. 

In the fourth step of the 
process, the participants consider 
five resources available to them^ 
when planning for action: force- ^ 
field analysis, management consider- 
ations, helping relationships, 
scientific knowledge, and self- 
initiation skills. The meaning 6f 
these terms is explai/ied iri the 
handout materials given to the 
participants. Exercises are provided 
that apply two of these resources, J 
helping relationships and management « 
considerations, to Mrs. Jones* 
problem. After some consideration 
of additional data-gathering tools 
and the dynamics of a small group, 
the participants enter on the final 
phase of the process: planning 
community RUPS improvement projects. 

The two follow up sessions are 
entirely concerned wi th these pro- 
jects. Participants meet to assist 
each other in analyzing their progress 
and plans. During these sessions, 
they explain their plans for the next 
steps of their projects. They crit- 
icize each other's problem state- 
ments, force fields, data-gathering 
designs and results, action plans, 
and outcomes. Criteria are provided 
for reviewing each other's work. 
During the second session, the 
participants also consider how to 
maintain change once.lt has been 
achieved and Improve the sharing of 
ideas and support. 

Another major area of concentra- 
tion in this product is teamwork 



skill building. Skills learned 
Include paraphras i ng/ gi vl ng and 
receiving feedback, recognizing one's 
own style of communicating, analyzing 
small group dynamics, develop'^ng 
supportive relationships, and group 
planning^ Instruction in communica- 
tion skills and the RUPS process are 
intermingled. Many of the exercises 
provide practice In both at the same 
time. For example. In one exercise 
team members work together--us I ng ' 
paraphrasing, listening, and helping 
behavlors--to develop a force-field 
analysis of their teamwork relatlpns,. 
The commual cation techniques are 
essentially those taught more inten- 
sively in Northwest Laboratory's 
I n te rpe rsona 1 Commun I ca 1 1 ons training 
package (see pages 32-37) . . 
( 

Trainee Assessment \ 

Brief multiple choice Vests 
follow each subset. Answer keys are 
printed on the reverse side of these 
tests. / 

For the most part, however, 
assessment Is Informa 1. Participants 
may assess their own understandjng ~ 
of the concepts by attempting to put 
them into practice, In the exercises. 
In each team, participants react to 
their teammates' performance in the 
activities. 'Each team. In turn. Is 
observed by another and receives feed- 
back on how It functions as a group. 

Product Development and Evaluation 

The materials were developed over 
a six-year period. The training design 
and Interpersonal content are In part 
derived from the work of the National 
Training Labdratory Institute for 
Applied Behavioral Science. Develop- 
ment of the materials was furthered 
by col laboratlon I n the* I nsti tute 's 
Cooperative Project for Educational 
Development (COPED). The first 
attempt at a complete design was made 
In collaboration with the National 
Board of Education of the Methodist 
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Church. The package was then used 
for training in the Brooklyn, Detroit, 
Jackson, and Livonia public school 
districts in Michigan as part of the 
COPED project. Next, a model of the 
classroom version was tested at three ' 
successive annual conventions of the • 
National Association of Classroom 
Teachers. Final revision of the 
Classroom Vers ion was accompl i%hed 
through cql 1 aboration with the 
National Education Association. 
Central Washington State Col lege, - 
and the state education associations 
of Oregon and Washington. The Admin- 
istrators' Version was then developed 
aniy^tried out at various sites in- 
^cKlKli'ng the Portland, Oregon, public 
school district and the University 
of I daho. 

The Afield test of the materials 
was conducted at five workshops In 
four western states. Participants 
at the workshops were chiefly 
administrators at one site, admin- 
istrators and teachers at two sites, 
and teachers and teacher Interns In 
the other two cases. A pre- and 
posttest was administered 'evaluating' 
conipetence In the use of force-field 
analysis, a, core skill of the pro- 
gram. A test of comprehension of 
problem-solving and communication 
concepts, the program focus, was 
given before and after training. 
Data from both measures 'indicate 
significant improvement in perform- 
ance. A fallow lip questionnaire on 
the degree of continued use of the 
process was *sent to participants 
six flionths after training. Greatest 
utilization occurred at the site 
^h^re participants trained as a task 
force working on a particular 
projject. At other sites, between 
one-*half and two-thirds of the 
participants continued to use the 
process. In response to a question- 
nalr'e concerning the worth of the 
training experience, highly positive 
reactions were expressed by adminis- 
trators and teaqhers; less favorable 
opinions came from some of the teacher 
Interns. The developers attribute 



the negative response to the fact that 
one of the intern groups had >ust 
experienced an In tens ive, unstructured 
T-group workshop. The stj-uctured RUPS 
training was Incompatible with the 
first experience and resistance was 
aroused. The developers recommended 
that for better participant satisfac- 
tion and results, the two types of 
experience not be Sequential. 

Contnent 

The fact that this program serves 
a dual purpose Is strongly In Its 
favor. Participants gain experience 
In both problem-solving and group- 
process skills In one workshop. 
The program has three^ther charac- 
teristics, however, that should be 
kept In mind by anyone who. Is consid- 
ering adopting It. First, the mater- 
ials and sequencing are highly 
structured. They may not be well 
received by persons who have had 
T-group experiences. Second, the 
program observes a "discovery" 
approach. Developers state that Its 
direction and content emerge gradually. 
Indeed, there Is no staterrjent of what 
the "research utilizing problem 
solving process" Is until the fourth 
subset. The "discovery" approach may 
not appeal to those with a low toler- 
ance for ambiguity. Third, coverage 
of the various phases of the problem- 
solving process Is uneven, with some 
phases receiving more emphasis than 
others. 
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Costs 




e 






Quantity 

Required {terns f needed. 


Cost 
•per Item 


Replacement 
rate % 




t 


Leader's Guide one per 

leader'" 






• 




Administrator's Version 


$8.75 


reusab le 


> 


( 


) Classroom Version 


8.50 


t 






Participant Materials one set 

per par- 
ticlpant 




consumable 






Administrator's Version 


6.15 








Classroom Version 


6.25 • 








Audiotape one per 
^ ' worl<shop 




reusable 






Administrator's'' Version 


3.^5 








Class room Version 


3.50 ^ . 








Text: Diagnosfng Prorv one per 

fesslonal Climate participant 
of Schools 






s 




Administrator's Version 


6.95 


reusable * 






Class room Vers Ion 


2.95 


reusab le 






-While It Is possible to have one experienced leader for 
.24 is the recommended group size. New leaders should not 
than 12 participants. 


30 parti cipants , 
be assigned mdre 
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EDUCATION FOR Tfi^ PEOPLE 



Content Foqub Community Involvement 
in goal setting. 

Type of ProduQt Guidelines, resource 
book, and training handbook for use 
in a workshop. 

Availability Since 1972."' 

Developer Joint Committee on 

Educational GoaU and 

Evaluation 
California Legislature 

Assembly, P.O. Box 83 - 

State Capl tol 
Sacramento, California 
9581^ 

Distributor Same as developer, or 

State^ Department of 

Education 
Of f I ce of Program 

Planning 
721 Capitol Mai I 
Sacramento, California 
958U 



\ 



Target Audience 

School personnel and commUhlty 
leci^ers who are Involved In Improving 
'the^r schools through better commu- 
nication, understanding, and agreement 
among all segments of the school 
and community. This Includes 
teachers, <)dmInlstrators, PTA members, 
teachers' associ atidn representatives, 
s tuden ts , pa ren ts , an d" s choo I emp I oy 
ees of diversified backgrounds. 

Goals and Objectives ^ 

On completion of training, partic- 
ipants will better Identify with 
school goals and appreciate and under- 
stanid the real task of the school; 
they* will realize {hat policy making 

• ^9 
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Is not solely dependent on hierarchical 
structures; they will realize that 
change sponsored by school -commOn I ty 
Involvement Is effective. 

Participants will become ac- 
quainted with a goal-setting process 
In which they can eventually Involve 
othene In their communities. 

Materials and Equipment 

The Education for the People 
package consists of an introductory 
booklet, a volume of guidelines, a 
volume of resource readings, and a 
training handbook that describes a 
specific program based on the general 
guidelines. 



The booklet expliiH^s .the ration- 
ale for the goal-setting program, 
outlines the step-by-step planning 
by which the school-communi ty arrives 
at consensus on goals of education, 
and establishes priorities for 
action. The booklet takes the 
process from the inital public hear- 
ings to the poi nt "at whi ch the gov- 
erning board of the \tate adopts the 
dec i's ions of the district. It is 
available in ei ther^ Engl i sh or 
Spanish. 

Volume I, Guidelines for Total 




Community Participation ih Forming 
and Strengthening the Future of 
Public Elementary and Secondary 
Education in California , suggests 
steps that districfts should follow 
to insure adequate community 
participation, and outlines the con- 
clusions reached by the California 
Legislature regarding this plan. 

Volume II, A Resource Book for 
School -Community Decision Making , 
contains a selection of articles 
and resource aids pertinent to the 
goal-setting process. This book Is 
to be used as a reference book. 

Training Handbook for School- 
Community Goal-Sd^tt Ing Is the result 
of a pilot project implemented by 
the Institute for Personal Effectiver 
ness In Children (San Diego) for the 
Joint Committee on Educational Goals 
and Evaluation. This handbook con- 
tains a suggested schedule for the 
training program and outlines the ba- 
sic requirements for a successful 
workshop . 

How to jUse 



This product does not offer a 
specific approach that can be adopted 
by every district or county. The 
guidelines and the sourcebook are 
exactly that, guides and sources. 
Each district, county, or state may 
esign Its own program to suit Its 
d^n needs. The Training Handbook con- 
tains a description of one such 



p-rogram. The progra'r^ outlined io a 30- 
hour training workshop that can be don- 
ducted over a four-day period or spread 
over a two-week period with no session 
being shorter than three hours, e.g., 
ten sessions over a two-v^eek period, 
Monday thrpugh Friday, between 7:00 p.m. 
and 10p.m. The program Is primar- 
ily for those people who will act as 
trainer/facilitators for goal-setting 
groups in varfous communities. Listen- 
ing as an effective communication tool 
Is emphasized in the first two sessions 
of the workshop. Task-oriented group 
experiences, games, role playing, and 
practicing communication techniques are 
activities In the small-group sessions. 
Approximately 30 participants are 
recommended for each workshop^. 

The tral ner/facl 1 I tator for the 
workshop must have had previous 
experience both as a participant in 
the goal-setting process and In leading 
groups of trainees. 

Trainee Assessmeat 

The trainees' assessment fn the 
program outlined by the handt^dok is 
derived from the Immediate feedback 
during the course of training. Review 
and evaluation sessions lasting' from 
ten to thirty minutes are conducted 
at the end of each day's activity. 
Qn the last day, 6ach trainee completes 
an evaluation qtJ^s t lonnnal re on the 
workshop or, alternatively evaluation 
is mgde by the group. 

" ' The ultimate evaluation is when 
the decl'slons made during thq wor1<- 
shop are adopted by the board of edu- 
cation and the school district as 
goals of education. 

Product Development and Evaluation 

In accordance with Assembly 
Concurrent Resolution (ACR) 195 in 
1969, the Joint Committee on Educa- 
tional Goals and Eval uat 1 on was estab- 
lished to appoint advisory committees 
of representative lay people to' assist 
the Legislature in recommending the 
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goals, objectives, and priorities of 
education for California. 

In May 1970 the; Joiat Commi^ttee 
presented its conclusion tha^ a 
successful goal-setting proce'^s 
requires extensive and intimate 
-Involvement of the public with stu- 
dents and educators at the local level. 
Thus, i'n July 1970 the Advisory 
Committee on Guidelines for Goals 
was established to deve lo|l gui del i nes 
for school districts on d^ermining 
local goals of education with broad 
community involvement. In June 1971 
the Advisory Committee submitted ^ 
progress report to the Joint Com- ^ 
mitteewith their recommendations for 
this project and enclosed Volumes 
f and I I of Education for the 
People for approval. In 1972 a 
pilot project was Implemented by the 
Institute for Personal Effectiveness 
in Children (San Diego) and tested 
in two schools, one elementary and 
one secondary. The Training Handbook 
for Goal -Sett i ng resulted from that 
pilot project. Education for the 
People was distributed to the school 
disti^cts throughout California, and 
several have developed their owrl 
programs bas^d on these materials. 
Several counties have developed their 
own programs, such as Goal Setti ngs , 
developed by the Chico Unified School 
District. A few counties have adopted 
successful .programs from other 
counties. For example, From Goals 
to Action and School' and Community : 
Partners in €ducation , developed by 
Fresno County, have been adopted by 
several other counties In California. 
Either of these programs Is available 
from the developers for a nominal 
fee. Each has a different approach 
to the same problem. 

Costs ' ^ 

Introductory booklet 
' (English or Spanish) $ .50 
Volume 1: Qui del I nes $ .50 
Vo I ume I I : A Resource 
, Book $2.50 



Training. Handbook Free^ , 

Commeht ' . * , 

r — ■ 

The guidelines set forth In 
Education for the People encourage a 
variety of approaches to achieving 
community involvement In the goal - 
setting process. The materials 
emphasize the Importance of each 
community as a unique enti ty. Califor- 
nia Ijas the advantage of having a 
number of communi ties' wl th distinct 
cul turaly^f+f ferences . Those who re- 
spondedAo the State Department of 
Education's request that all school 
districts estabJ Ish educational goals 
developed thel.r own programs of imple- 
menting school -communi ty partnership. 
Here ard three examples: j 

The Training Handbook , distributed 
by the Joi nt Comml ttce on Educational 
Goals and Evaluation, suggests a four- 
^ day workshop of approximately 30 hours 
with around 30 participants. The 
participants build up trust and con- 
fidence In each other as individuals 
before the actual goal-setting train- 
ing begins. They are trained to con- 
duct more workshops following this 
format. 

School and Community; Partners 
in Education , developed by the Fresno 
County school district, forms a 
steering committee of 2k to 30 persons 
who are responsible for selecting 
participants for a community conference 
This program Is designed for a large 
gathering — a minimum of I00«peitsons-- 
for a 9:00 a.m. to 2:00 p.m., one-day 
session, or alternatively, tv^o evening 
sessions. The activities arc faster- 
paced and more structured than are 
those suggested In the Tral nl ng Hand- 
book . At the end of the session the 
participants are Informed that th(^ 
will receive copies pf the ned^fe and 
goals statements. The sorting and 
classifying of responses Is the 
function of the steering committee. 

, Goal Set ting I developed by the . 




Chico Unified School District, Is 
another type of program. Their 
steering committee designed a three- 
part approach. First, the dlstrict*s 
professional- staff is as'ked for their 
Input. Second, contact groups, 
comprised of PTA, Chambe^r qf Commerce, 
and repj;esentat i ves off the St/ate 
Unl vers I ty^maket re^commendatlohs . 
Third, after the educational goals 
recommended by these organizations 
have been implemented, the steering 
committee mails out questionnaires 
to registered voters in the community. 
The quest lonnai res request that ^ 
community members rank the go*als by 
priority and evaluate the district's 
effectiveness in implementing these 
goals. The data collected from this 
survey serves many admfnlstrati ve 
functions other than the goal-setting 
process. Thus, the Education for' the 
People package could be more correctly 
regarded as either a program for 
training or a resource for training. 
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EVALUATION WORI^SHOP I 

An ■ 

Orientation 



( 



Y 



content Focus Basic evaluation, ski 1 Is. 

Type of Product two-day workshop. 

Availability ' Since August 1971. 

Developer Stephen P.^ Klein and others 
Center for the Sti^dy of 
- Evaluation ^ 

Graduate School of Educatiori 
UCLA 

Los Angeles, California 
' ' 90024 

Distributor tJB/McGraw-Hi II 

Del Monte Research Park 
Monterey, California 
a3940 



Target Audience - . ' - 

This product was developed for 
teachers, administrators, and other 
, personnel involved in educational de- 
cision making. The distributor has 
suggested the workshop J^e presented - 
to a minimum of 24 persons and a 
maximum of 60, -Only person%^ho 
' have successfully complet^iT ^he work- 
shop are authorized to purchase the 
'^^'^rpduct and conduct the training.' 



Goals and Objectives 



The v^orkshop is directed toward 
two manor goals: (l) participant un- 
derstinding of tha kinds of information 
kn ev;^*tJiation can provide for education- 
al decision making and (2) parti ci pant 
understanding ot the general procedures 
and problems involved in selecting, 
collectf^ng, analyzing, and reporting 
that infornpat ion. Th^degree to which 
participants develop such .understand- 
ing will be indicated by their attain- 
ment 6f the fallowing objectives: 



sequence the major evaluation 
activities; 

Identify appropriate data selec- 
tion, collection, analysis, and ' 
reporting procedures for each . 
major evaluat4on activity; ^ 

Identify proper and improper use 
of evaluation techn i ques , methpd- ' 
ology, and design; 

Identify the kinds* of information 
that should be generated by each 
evaluation activity; 

Identify the kinds of information 
that are needed before evaluation 
decisions can be made; 

Identify the specific functions of 
the evaluator and those of the pro- 
ject director in each evaluation 
act i vi ty . 



Nante, describe, and properly 



Materials and Equipment , ' 

The workshop includes ^he following 
materials : 

Leader's Manual describes step-by-step 
how to organize and conduct the workshop. 
It includes verbal instructions to the par- 
ticipants, provides the leader with the nec 
essary material to cover in discussion 
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periods, and outlines procedural tasks 
^in handling the workshop materials. 

A Conversations Tape contains a 
series of conversations which rorm 
part Qf the instructional material > 
presented to the participants. -If 
the leader wishes, the participants 
can read the script of each conver- » 
sation. Copies of each script appear 
in-Appendix 1 of the Leader's Manual . 

A Participant ^s ljotebook cqptai^ns 
instruct ion. plus activities on five 
evaluation phases used in educational 
decision making, and, following the 
workshop, can be used as a general 
reference tool and guide to evalua- % 
tion practices and activities. 

Exercise Materials give practice 
jn the various evaluation activities: 
There are ten exercises plus answers, 
^am members are provided with group 
and individual answer sheets. Pre- 
and postte§ts are used* at the 
beginning ^and end of the workshop 
to measure the product. The responses 
front these tests and the postquestion- 
naires are sent back to the distributor. 

How to Use 

The EW| materials are used in a 
two-day workshop (8:30-if:00 each day). 
The distributor requires that .EWI 
materials b# purchased and used only 
by someone who has participated in 
the training. A^orkshop leader or 
coordinator must therefore already ' 
have taken tfie training program as 
a participant and be acquainted with 
the procedures and content. The 
coordinator has the Leader's Manual, 
which he or she should read before 
the workshop sessions. It contains , 
instructions on how to recruit and 
arrange and conduct the workshop. 
The leader is instructed to ^ead an 
excerpt. from the manual that describes 
the evaluation model on which the 
workshop was^ des i gned: "The Center's 
Changing Evaluation," by Stephen Klein, 
Gary Fens t^rmacher , and Marvin C. Alkin 
(from Evaluation Comment . January 
1971). The leader administers the 
pre- and posttests and must send the 



• answer sheets back i?q the distributor. 

To achieve its goals, this work- 
shop has each participartt play the 
rol^ 6f the evaluafbr in a simulated 
evaluat^ion of a tenth-grade biology 
program. The participants, working 
in teams of three, conduct the needs 
assessment for the program, plan and 
review the evaluation procedures for 
the program, assess" whether it was 
implemented properly, determine what 
might be done to improve the program, 
assess its merits and final results, 
and report the evaluation results to 
lay audiences. Participants learn 
dbout each of thes^ eval uat ion acti- 
vities by means of a four-par t process : 

(1) Instruction:^ exposure to 

learning materials 

(2) Practice: 'performance of 

evaluation tasks ; 

(3) Feedback: information on 

how the evaluation should 
have been performed; 
(^) Discuss ion : presentati on 

of various viewpoints toward 
the feedback materials. 

In seven of the ten exercises, 
the members of each team work together • 
to solve evaluation problems. For 
these exercises, there Is one team 
answer sheet with both directions and 
answer ^aces and three individual 
sheets with directions only. These 
seven exercises are designed to provide 
practice in cooperation and communica- 
tion during the evaluation process. 
The three remaining exercises are done 
individual ly. 

Trainee Assessment 

There is no formal trainee assess- 
ment. The exercises provide practice 
and feedback. The pre- and posttests 
contain 35 objective true/false and. 
multiple choice items concerning the 
roles that evaluation plays in planning, 
implementing, and assessing programs 
and projects covered in the workshop 
training. Because the tests are designed 
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only to measure the effectiveness 
of th'e training program itself, the 
results are not given to the parti- 
cipants. The leader is directed not 
to report, individual scores to the 
participants, although they may be 
informed of the distribution of test 
iscores I f the leader wishes. The , 
allotted pretest time is 20 minutes; 
, post test time, 25 minutes. No info^- 
matibn is pravided on the reliability 
or validity of the tests. The^p^omo- 
tiorfai flyer des.'cribing the product 
does not mention the pre- and post- 
tests. 

Costs 

For anyone interested in attend- 
ing a workshop run by CTB/McGraw-H i 1 1 
in, order to qualify as a leader, the 
cost of attendance ' i s $100.00 per 
person plus traveling expenses. For 
this fee, CTB/McGraw-H i 1 1 provides the 
leader and all materials for runnfng 
the trai ni ng. 

If ^ qualified leader wants to 
run a worl<shop, the cost of the ' . ' 
Leader Manga 1 , Including the Con- 
versation Tape , is $40.00. The 
Participant's Notebook and Exercise 
Materials cost $40.00 per participant. 
A 25% discount is civallable on orders 
of 30 or more . 

Product Development and Evaluation 

Beginning in^ 1969, this product 
was developed at the Center for the 
Study of Evaluation, UCLA, under. the 
direction of Dr. Stephen P. Klein. 
The work was sponsot'ed under a grant 
from the USOE. Three feasibility 
studies were conducted with a pilot 
version between March 1969 and 
April 1970. A revised version was 
rield-tested at five sites during / 
the spring and early summer 'of 1970. 
Participating in the field tests were 
school administrators project di r- 
ectors, and curriculum supervisors. 
The results were positive, but indi- 
cated the amount of reading time should 
be reduced and the participants given 



a larger role in the workshop. Other 
/changes included modifying the manuals 
to three-ring looseleaf notebooks, ex-' 
panding the context of the workshop to 
include- reporting Vesults , to lay 
audiences, and changing the length of 
the workshop to two days . A revised 
version was field- tested at two sites 
at the end of summer 1970. Partici- 
pating were staff from developmental 
centers, USOE personnel , and representa- 
tives of the National Science Foundation, 
-all of whom were knowledgeable in eval- 
uation. The results of these field v 
tests brought changes in some of the 
exercises. Again revised, the workshop 
was tested at 12 operational field test 
sites throughout the United States be- 
tween Oatober 1970 and August 1971. 
Xhis field test was for school and state 
department of education .personnel . 
The results were positive." The product 
was then revised and edited by McGraw- 
Hill into a final version by August 1971. 

Comments 

The program will not train the 
participants to be eval uators , but It 
will provide a fund of information for 
those with 1 i tt le background ine^valu-^ 
ation.* Participants learn how evalua- 
tion can establish a better base for 
decision making through all stages of 
instructional program selection, plan- 
ning, and adoption, they discover how 
evaTuat-ion can help make programs 
effective, rather than sjmply measufe 
effect i veness after the fact. Through 
the s Imulatlon exercises placed through- 
out the materials, the participants "get' 
the feel" of the evaluator'-s duties. 
They also- learn what the administrator 
may reasonably expect of the evaluator 
and what measures should be taken for 
optimum benefit from the evaUiator's 
services. Among those seryjtes is 
assistance in avoiding qorv^rontat ions 
with the community over unpopular 
testirig instruments. Others are coordi- 
nating installation of new programs and 
monitoring the progress of the^e 
programs. 
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DESIGNING INSTRUCTIONAL PROGRAMS 



Content Focus Analysis of factors 
for effective program planning; 
opportunities and problems in 
curriculum design. 

Type of Produot Training package 
designed for team activities in 
workshops, college courses, and 
school Sjett i ngs . 

Availabilitij. Fall 197^. 

Developer Far West Laboratory for 

^Educational Research and 
Deve I opment 
1855 Folsom Street 
San Francisco, California 
9^103 

Distributor Same . 



Target Audience 

The intended users of this pro- 
gram are inservice school personnel: 
teachers, building principals, de- 
partment heads, curriculum coordina- 
tors, ass istant superintendents , arid 
preservice graduate students in 
educational administration or curric- 
u I um p I anni ng. Others involved in 
program planning, such as parents-, 
students , or communi ty representati ves , 
may participate if the situation 
permits. 

Proficiency in or knowledge of 
i ns truct ioria I planning and management 
is not required for participation. 

Goals and Objectives 

This program intends to improve 
school staff practices in designing 
i ns.truct ional programs and selecting, 
appropriate curricula by: ^ 

gaining familiarity with a 
format for'^ des i gn i ng curriculum; 



analyzing basic learning assump- 
tions, and matching programs 
with intended learning outcomes; 

selecting curricular materials 
appTtcable to selected class- 
' room prq,cedureS ; 

comparing and analyzing specific 
features in different curricula 
for appropriateness; 

determining feasibility of a 
^ given program. 

Materials and Equipment 

The Designing Instructional 
Programs uni t is divided into fiVe 
modules, printed" in two participant 
handbooks. The training material corr- 
tained in each module includes exer- 
cises, techniques , procedures , and 
tools to help the participant perform 
sets of tasks critical to program 
design. 

'"Chalk Talk,'' used in Module I , 
is an educational game consisting of 
a set of cards, the playing board. 
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pieces in three different colors , a 
set of rules, and a glossary of 
frequently used educational terms; 
it outlines program design procedures 
and helps motivate participants to . 
proceed. 

Modules 2 thrOugh.5 use fictional 
situations and accompanying exercises 
(both individual and group) to aid 
partTcipants to understand and ac- 
quire skills according to each 
module's specific goals. 

The Coordinator's Handbook 
comp 1 e tes the set of materials for 
this unit. It is a step-by-step 
guide for administering the materials 
for up to 12 participants* 

How to Use 

A coordinator is essential in 
several discussions during the 
training. The primary criterion for 
this role is thorough familiarity 
with the materials. The training can 
be offered by school district staff, 
university faculty, or ex^rnal 
training consultants, 

"Chalk Talk'' is the central 
activity of Module I. In this game, 
participants perform most of the 
critical tasks of program designing 
by facing a range of poss i b le opt ions 
and some potential problems. By 
referring to the Glossary, they learn 
the meaning of educational terms 
applicable to the current situation. 
The game is played by three teams 
of three or four persons who select 
a subject and grade level, then 
decide how the subject may best be 
taught.^ During the game, the coor- 
dinator is a neutral observer and a 
game leader*. If coordinating 
assistance is available, more than 
one game can be played simultaneously. 

Module 2, Analyzing Basic Assump- 
tions , provides six primary questions 
to ask and four parables from which 
trainees prepare a set of basic 
assumptions which form the 
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^philosophical basis for their 
programs . 

Module 3, Matching Programs With 
Goa I s , consists of three parts: the 
first discusses the relationship of 
four types of learning goals to each 
other; the second consists of a tool 
for sorting and grouping learning " 
goals into four types, deciding 
whether to revise or not, then matching 
them with appropriate program design 
options; the third is another tool to 
help match one type of learning goal 
to specific design options. 

Module A, Assessing Resources and 
Constraint^ , trains participants to 
assess the readinesj of the school 
system to implement new or revised 
instructional programs at the building 
or district level. The participants 
are provided with written materials 
and exerci ses that focus on problems 
of resources and constraints such as 
costs and time. 

Module 5, Selecting Curricula , 
requires participants to examine two 
sample curricula. * The "Curriculum 
Analyzer" is a tool intended to help 
sort through and rate alternative 
curricula according to criteria 
deemed critical by participants. 

The developers of the unit rec- 
ommend from 6 to 12 participants for 
"Chalk Talk" and from 6 to 30 for 
Modules 2 through 5, A single coordi-. 
nator can work with these numbers most 
effectively. ^ 

Each module requires approximately 
three hours to complete, but the total 
training time varies according to size 
of the group and the setting. A total 
of 15 to 18 hours training time has 
been estimated but is subject to ad-, 
Justment. As an alternative method of 
scheduling, Module I can be completed 
by a group at one $itting. The intro- 
duction and individual exercises, \h 
Module 2 can be completed in a group 
or outside the formal training environ- 
ment, although the team exercises must 



be completed in team settings. 
Module 3 can be completed either indi- 
vidually outside tKe training sess ion, 
or on a team basis. Module ^ can be 
completed In or ol t of class, Indi- 
vidually or on a tsam basis. Module 5 
should be completecLJn class. Ideal 
scheduling of the unit would be five 
sessions, two or three times weekly, 
for three-hour periods. 

Trainee Assessment 

Self tests with suggested 
responses are provided throughout 
the program for trainees to evaluate 
their progress and general knowledge. 
Evaluation of individual performance 
is done by other team members. At 
the end of each activity contained In 
the modules, teams may discuss 
problems or questions that have- 
arisen during each session. 

Product Development and £va1 oration 

Components of the materials vyere 
Initially developed and tested sep- 
arately. Modules 1 and 2 were tested 
on at least three separate occasions 
as prototypes, and have since been 
tested five times with various types 
of' school user groups. Modules 3i 
4, and 5 have been tested five times 
in workshops and local school 
districts. 

Test sites for these materials 
were as follows: a 15 hour summer 
session workshop In school adminis- 
tration, two 15 hour university-level 
Inservlce workshops, and at least 
five other workshops Including school 
district workshops which used the 
materials In several time configura- 
tions. The total number of test 
participants were approximately 200. 
The materials have also been Independ- 
ently reviewed by experts In curric- 
ulum and Instruction. Information on 
field test data Is available from the 
developer. 



Costs 

A full set of the materials 
Including the ''Chalk Talk'* game suf-^ 
ftclent for ten participants and a ' 
coordinator Is priced at $135. Thus 
cost per trainee equals $13.50. 
Additional participant materials cost 
$11.75 for each pair of participant 
'manuals. A check or money order must 
accompany each order. California 
residents should prepay applicable 
sales tax. 

Comment 

After this training, participants 
should have a clearer awareness of their 
own basic assumptions, an awareness 
they can continue to draw on y^hen con- 
sidering programs In the school. They 
have learned to develop programs having 
Internal consistency, to recognize the 
Implications of selecting different 
Instructional programs, and to see the 
necessity for compatible components. 
They have become familiar with methods 
of determining how the Institutional 
and community climate Is going to 
affect adoption of programs. 

A major asset of the program Is 
the direct. Immediate applicability 
pf Its procedures and tools. The tool , 
for classifying goals and matching 
program outlines with goals can be used 
'either for deriving new goals or work- 
ing with those already adopted. The 
'^Curriculum Analyzer" makes It easier 
to be certain that all variables are 
considered; It can be used to order 
preferences for a curriculum, then . 
rate different curricula on how well \ 
they meet these preferences. The tools 
for cost analysis and budget adjustment 
follow rational Indexes so that they 
can be used on the job. 
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CSE ELEMENTARY SCHOOL EVALUATION KIT: 



jvl^ds 
Assessment 



u 



Target Audience 

The KIT was specifically de- 
signed for elementary principals or 
superintendents; it could also be 
used by curriculum planners, educa- 
tional consultants, teachers, school 
boards, parents, and any other de- 
cision makers at grade levels 1-6. 

Goals and Objectives 

The evaluation KIT series 
designed by the Center for the Study 
of Evaluation (CSE) are all based on 
the following definition of evaluation: 

Evaluation is the process of 
ascertaining the decision areas 
of concern, selecting appropri- 
ate information, and collecting 
and analyzing information in 
order to report summary data 
'useful to decision makers in 
selecting among alternatives. 



Content Poaus Selecting educational 
goals • 

Tifpe of Pvoduat Materials that provide 
methods and ^procedures . 

Availgbi litij S i nee 1972. 

Developer Ralph Hpepfner and others 
Center for the Study of 

Eva 1 uat ion 
Graduate School of Education 
UCLA ^ 

Los Angeles, California 
90024 

Distributor Al lyn and Bacon, Inc. 

Longwood Division^ 
Rockleigh, New Jersey 
07647 ^ - 

The needs assessment KIT , which is the 
first in a series of school evaluation - 
KITS , provides elementary school prin- 
cipals, with systematic procedures for 
maki ng i nformed , rat i ona 1 deci S ions 
about the goals most appropriate for 
their schools. These procedures fall 
under the general title of need^ 
assessment. The goals of the program 
are to: (l) provide procedures for 
gathering infof^mation about goals the 
school should be meeting; (2) instruct , 
the principal in selecting tests tb 
measure student performance on highly 
rated goals; (3) provide the principal 
with an effective way of interpreting 
the school's test scores in relation to 
those of other schools wi th" s imi lar 
characteristics; and (4) show the 
principal how to use a decision model 
to transform the information already 
gathered into a set of critical need 
areas for the school. After completing 
the, KIT , the user should be able to make 
meaningful, explicit assessments of a 
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school's educational needs. 

Material's and Equipment 

The Gui debook is essential to 
use of the KIT . It gives the user 
the rationale for needs assessment 
and provides sti(ep-by-step procedures 
for determining^he school's educa- 
tional needs. It\ describes in detail 
the three methods\of determining goal 
priorities. These three assessmen^t 
methods are the bis is of the KIT . 
The Guidebook -^out I ines cri ter ia by 
which test instruments can be selec- 
ted. The appendix lists the names of 
^mmonly used achievenfefit tests and a 
four-point evaluation jnethod compiled - 
by Independent test experts for each 
test. The Guidebook also provides 
advice on how to order and administer 
achievement tests and how to analyze 
thei r results. 

Principal's Goal Rating Forms 
provide a list of gpals and spaces 
to rate each goal. There are tvio 
different forms for each\f th^g sfx 
elementary grades: 

Te^^jchers' and Parents' Card Sort 
Gaal Assessment Set includes 
\ ^0 decks of I06 Goal Cards; 
Each card describes a dif- 
ferent goal ; 

10 sets of Rating Mats (5 per 
set) marked "Unimportant," 
"Marginal Importance," "Aver- 
age I mportance "Moderate 
Importance," anrf "Most 
Important"; \ 
50 Rating Forms/ for tallying 
the goals. v 
Parents' Goal Rat i n\ (Jues t i on- 
nalre (^8 copies) contains the 
same I06 goal statempnts presented 
on the Goal Cards. /Both the 
Assessment Set alfTci the (iuestion- 
nai res i nc I ude i ns^f ruct i ons on 
thel r use. 

How to Use 

The Gui debook is the essential 
component of the product. It is a 
manual/textbook providing instruction 



in evaluation methods for either a 
single decision maker to rate goals 
Individually or for using input from 
teachers, parents, and community 
members. The Guidebook instructs a. 
principal In how to distribute and num- 
ber the Questionnaires and Goal Cards 
to obtain this input. Procedures are 
described for rating the goals at each 
grade level. Chapter 3 provides the 
decision maker with procedures for 
selecting and evaluating testing in- 
struments to measure skills that 
correspond to the goal statements. 
Chapter k describes procedures for 
selecting and obtaining standardized 
tests, two methods for administering 
tests, determining the frequency of test 
administrations, and organizing and 
interpreting the analyzed data. Chafer 
5 details procedures for selecting 
critical problem areas and goals based 
on earlier selection and rating of goals 
and results of measuremerit techniques . 
Finally, the chapter describes yarious 
methods and factors to consider in 
communicating the results of the. eval- 
uation to students, teachers, parents, 
and school board members. 

The parent, teacher, and cdmmunity 
input for choosing and ranking Jo^ls is 
obtained through the Goal Cardsjand 
(luestionnai res . 

Teachers' and .Parents' Card Sort 
Goal Assessment Set is used by the 
parents or other advisory group. They 
sort the cards according to the impor-' 
tance each person places on each goal 
and then put these cards on the corres- 
ponding Rating Mat. ^ . 

The Parents' Goal Rat fng Quest ionnai re 
can be distributed by mall. Parents re- 
spond to the items and return them to the 
admi nistrator. 

Trainee Assessement 

The product is a support tool for 
the principal/superintendent. The KIT 
does not address itself to "trainees." 
There is no assessment instrument. 
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Product Development and Evaluation 

.The KIT was developed by the 
School Evaluation Program at the 
Center for the Study of Evaluation 
at UCLA assisted by several elementary 
schools, lyo/E/A (Institute for De- 
velopment of Education Activities), 
and CESAA (California Elementary 
School Administrators Association). 
Several development personnel were 
former elementary school teachers 
and a/Jml nist rotors. 

The Information in the KIT was 
based on Input from elementary school 
principals and superintendents. The 
components of the KIT were tried out 
with educators at national meetings 
and with 23 principals and superin- 
tendents in. Gal Ifornla. The major 
field test of the KIT was conducted 
during fall 1971 iTTSs schools 
across the country arid 103 schools In 
California. The following conclusions 
resulted from the field tests: 
The KIT will continue to be 
addressed primarily to prin- 
cipals because they are the 
main decision makers in se- 
lecting educational programs. 

Principals found the KIT useful 
because they do feel pre^ssured 
to make realistic needs assess- 
ment; and because the Kl T ' s 
card-sort procedures assist 
principals In Involving teachers 
and parents In making their 
decisions. 

The KIT provides useful Informa- 
tion regarding test selection 
and evaluation and the developers 
wi 1 1 expand the KIT to make test 
results more useful to principals. 

The goal statements seemed to be 
comprehensive and effectively 
organized, but parents and com- 
munity members had difficulty 
with the vocabulary. (Since the 
time of that field test, the 
language of the goal statements 



has been revised and tested with 
a group of parents from low and 
middle socioeconomic levels.) 

Some changes were recommended for 
the card sort procedures , e.g., 
number of cards. Illustrated 
Instructions, and expanding part 
of the card sort process to 
present questionnaire. 

Costs 

The complete KIT costs $lU.95. , 
This Includes the Guidebook ; 2^ copies 
(^ for each grade level) of the Prin- 
cipal's Goal Rating Form; ^8 copies of 
Parents* Goal Rating Questionnaires; 
10 decks of Goal Cards (106, cards per 
deck); 10 sets of Rating Mats (5 per 
set); and 50 Goal Card Tally Sheets. 

One complete Kl T is available on 
a 30-day approval basis. Individual 
components of the KIT are available in 
the following lots and prices: 

(^8 copies) Questionnaires $ 8^95 

(2^ copies) Principal's Goal 

Rating Formic $ 6.95 

(50 Copies) Goal Card Tally 

Sheets $ 6.95 

(10 sets) Rating Mats and Goal 

Cards $29.95 

The Gui debook Is not distributed 
separate ly . 

Comment 

It Is necessary to examine the KIT 
to understand whether and how to use 
it. The essence of the Instruction Is 
the Gui debook , which is an operating 
manual or textbook. This book is very 
useful to principals as a reference or 
support tool. From a user viewpoint. 
It is unfortunate that the Guidebook 
is not sold separately. Th^ complemen- 
tary materials (Parents ' Que^tionnai res 
and Goal Card Set) appear to be useful 
tools for getting the school and com-' 
munlty involved In determining educa- 
tional goal s . 
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EVALUATION FOR PROGRAM IMPROVEMENT 



Content Foqus Program evaluation 
and its relation to specific roles 
in the school community. 

Type of Product Materials fbr team 
activity in workshops. 

Availabilitif Fall 197^*. 

Developer Far West Laboratory 
1855 Folsom Street 
San Francisco, California 
9^103 

» 

Distributor S ame . 



Target Audience 

This product is designed for - 
school personnel interested in pro- 
gram evaluation: specifically, 
teachers, counselors, department 
heads, curriculum directors, princi- 
pals, vice principals, superinten- 
dents, and directors of special pro- 
grams; also, parents or community 
members interested in program manage- 
ment may find th^ unit of value.. 

Goals artid Objectives 

This product is designed to 
help instructional program managers 
develop the knowledge or skills neces- 
sary to identify, collect, organize, 
and analyze information useful in 
making decisions about program modi- 
fication. The unit is intended* to 
teach participants knowledge and skills 
for conducting program evaluation; 
identi fyi ng^ necessary and feasible 
decisions to be made about program 
modi f i cat i on ; i dent i fy i ng al ternat i ve 
courses of action; determining rel- 
evance of information to modification 
decisions; and using information about 
program effectiveness to modify i^n- 
structional programs. The product is 



intended to avoid involving participants 
in overly theoretical concepts or tech- 
nical terminology. ^ 

Materials and Equipment 

The Evaluation for Program Im- 
provement unit consists of a Partici - 
pant's Handbook , a Coordinator 's 
Handbook , a series of transparencies, a 
filmstrip, and a cassette tape. 

The Participant's Handbook includes 
directions, supporting material, read- 
ings, worksheets for participants to 
use as they go through the activities, 
and an annotated bibliography. The 
wqrkshop is divided into five sessions, 
and this organization is reflected in 
the handbook. Activities are structured 
to provide the participant with a basic, 
understanding of what is involved in 
evaluating an ongoing program. The 
first workshop session provides an 
orientation to the total process of 
program evaluation. Sessions I I through 
IV feature simulation exercises in whj ch 
the participants' take the roles of 
evaluation committee members and attempt 
to evaluate a program; these activities 
are followed by feedback and di'squssion. 
Sessions II, III, and IV are followed 



62 

64 



^ by reading assignments to be com- 
pleted before the next session begins.. 
All materi-ais for the worlcshop are 
contained in the Part i cipant ' s 
Handbook . 

In addition to providing guide- 
lines for conducting the workshop, 
the Coordinator's Handbook also con- 
tains wr^i tten narration and general 
instructions for group discussion. 
Excerpts from the Parti cipant's 
Handbook are reproduced in reduced 
type in the Coordinator's Handbook , ' 
so that the coordinator need work 
with only one document throughout 
the works^p. 

Transparencies, a color film- 
strip, and. a cassett^e audiotape are 
provided with the unit. 'The informa- 
tion they present is summarized in 
the Coordinator's Handbook and Par- 
tlcipant's Handbook . The (^ordi na- 
tor's Handbook describes alternatives 
if equipment is not available. 

How to Use 

Each of the five sessions in the 
Evaluation for Program. Improvement 
unit takes approximately three hours-^ 
to complete. These sessions can*^e 
scheduled as one session per week 
for five weeks; one session pier day 
for five days; one weekend (ifrcluding 
Friday afternoon or evening), or ^o 
weekends. The Coordinator's Handbook 
also contains suggestions for con- 
ducting the unit In larger or smaller 
amounts of time. 

The coordinator Is responsible 
for guiding the participants through 
the activities. The Handbook ass ists 
him or her in facilitating logical and 
improvi sational thi.nkina on the part 
of participants. A background in 
educational evaluation oi\knowledge 
of role playFhg and simulation will 
be helpful to the coordi natoT but is 
not absolutely necessary. InNuldltlon 
to reading the assignments, himX^r 
(lerself, it is recommended that th>a 
coordinator collect as many books \ 
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listed on the annotated bibliography 
as possible, read the chapters specified, 
and have the books available for loan 
to interested participants. The 
Coordinator's Handbook contains all in- 
formation tieeded to conduct the workshop. 

The coordinator should plan to 
spend about 15 hours preparing for the 
workshop. Additional time may be needed 
to arrange for facilities, released time, 
substitutes, etc. Workshop planning 
should probab ly begi n a month before 
the scheduled dates. ^ 

The ynit is designed arou/id an 
actuaU case study in which-a committee 
planned the evaluation of a district- 
wide English program. Participants are 
assigned the roles of evaluation team 
members and are given instructions 
about how to play their roles; the team 
is given instructions about how to 
attack the problem of planning the 
^Valuation design. The team progres- 
$ively defines the evaluation problem 

it meets in corfsecutlve trainLng 
Sessions. Thus, the participarfts inter- 
act in a simulated setting which, is 
similar to one they might face In their 
own work. The role descriptions are 
Included In the Handbook and each par- 
'ticipant Is requested to select the 
role he or she would feel most com- 
fortable playing, and also to Indicate 
second and third choices. The coordina- 
tor shoujd take particular care In 
assigning the role of the educational 
psychologist serving as part-time eval- 
uation consultant. This perjson should, 
if possible, have some background Jn 
educational evaluaticfn. . If not, he or 
she should read through the entire 
Participant's Handbook ahead of time, 
read some of the reference materials , 
and meet with the coordinator to discus^ 
the thrust and specific concepts of the 
unit before beginning the simulation 
exercise in Session II. 

The amount of space needed depends ' 
on the number of participants. The 
workshop can be conducted by one coor- 
dinator with between 6 and 25 partici- 
pants. However, If there are more than 



11, the group wi I I vhave to be divided 
In two. During theXsimulatlon ex- 
ercises, each of theVe groups will 
need an area , where i tXcannot hear the 
other groups. Audiovisojal equipment 
should be available forNuse with the 
fllmstrip and transparenoJes , 

Trainee, Assessment 

An optional b i ograph i ca nforma- 
ti\Dn form, questionnaire, andfprelim- 
Inary exercise are included in, the 
Coordinator's Handbook (Session I). A 
final questionnaire and exercise are 
also included (Session V) . The pre- 
liminary instruments may be used as a 
pretest to help the coordinator ass^n 
roles. Participant copies of these 
li^struments are not available. The 
coordinator is encouraged to reproduce 
therrKfrom the copies in Hi is or her 
Handbbok if hp or she desires to use 
them. 

The t rai nee^Vrecei ve immediate' 
feedback via the djseu^s ions fol low i ng 
each exercls^ affa the crTttq^ies of the 
actual evaluation the simulation is 
based on. 

Product Development and Evaluation 

Evaluation for Program ' I mprovement 
i s one of a series of training units " 
developed by Far West Laboratory to aid 
schools in clarifying purposes of in- 
structional programs, desisning the 
programs, and obtaining evaluative in- 
formation. This unit was developed in 
1973 based on a limited but intensive 
study of the "needs of school instruc- 
tional staff in th6 area of program 
evaluation. A prototype version of the 
unit was tested in a university course 
during the fall. of 1973, and a revised 
prototype was tested using school 
teachers in the spring of 197^. The 
main field tests of the unit were con- 
ducted In the summer and fall of 197^. 

The unit has been tested with mem- 
bers of the target audience in a variety 
of settings: at San Francisco State 
University, an ACSA leadership 



conference, and several schopl districts. " 
Almost without exception, participants 
have agreed that the unit addresses a 
definite need, and that it is a worth- 
while and enjoyable experience. After 
completing the unit, most participants 
have demonstrated an understanding of . 
Its approach to program evaluation. 

Cost 

The Participant's Handbook is $7-95. 
The Coordinator's Handbook and a ful 1 set 
of audiovisual materials are sold to- 
gether for $3^.95. A'check or mqney 
order must accompany each order.' Califor- 
nia residents should prepay applicable 
sales tax. 

Commenf^ 

This unit adequately presents e'valu- 
ation as a process through which one 
assesses progress toward purposes and 
objectives, and the degree to which they 
havj2 he^n achieved. This material will^ 
provide training and experience in per'- 
forming the evaluation function of pro-* 
gram management, or give a betteJ^under- 
standirtg of what is involved in evaluating 
-*an ongoing program to any participant, 
despite his or her initial educational 
background. The unit does not attempt to 
turn part i'ci^pants into professional evalu- 
^tors, buf to arouse their awareness of 
tfj|e pcftt played in evaluation by different 
members of the" school community. The 
developers realize that very few program 
modification decisions of any importance 
can be classified as completely right or 
wrong. / However , the developers expect 
that users of this training material will 
be more likely to make decisions which 
contribute to an ultimately positive effect 



Evaluation for Program Improvement 
I s a comprehens I ve , sel r-conta l ned t ral 
Ing unit. If Is also part of a series 
training materials whTch Includes 
DetermI nl ng I ns truct ional Purp,oses , 
Pes I gnl ng I njst ruct lona 1 Programs , and 
Planning P rogram I mp 1 ementa 1 1 on . 
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SHARED DECISION MAKING 



Content Foeue Training In making • 
cooperative decisions. 

Ty pe of Produat Traln4ng W9!:kshrops 
ana discussion sessions. 

-Avai labi Hty Now; first published 
June 1973. 

Dgvg loper^ Raymond G. Melton 
n Rodger E. Cryer 
National CI uster 
f - Coordination Center 

^ ' .Florida Department of 

Educat Ion 

128 Milt Johnson Blvd. 
^ Tallahasse, Florida 

3230i» 



Dietributor 



Association of 

Cal I forn la School 
^ Administrators 
1575 Old Bayshore 
"Highway 

Burllngame^ California 



Target Audience 

The Shared Decision Making ( SDH) 
program Is part of the Professional 
^development Program of the Association 
of California School Administrators 
(ACSA) , whose general target audience 
Is school administrators. This 
particular program Is Intended for 
administrators and teachers at all 
grade levels fn districts or schools 



that have a gredt deal of parent 
participation- 



Goa\jS,and Objectives 

The underlying assumption of. this 
training program Is that If those 
affected share decision making, they 
can assume greater responsibility for 
decisions, make wiser decOsIons, and 



be more committed tQ^the decisions made. 
The developers seek to familiarize ^ 
partlcl|jants with relevant resjearch * 
and . I nformat Ion ^" Individual and group 
accountability and responsibility, and 
a system for obtaining group support. , 
They see the workshops and discussion 
papers as providing skills 'and methods 
that will aid -in the developmerit of 
prptotype programs for shared <JecIslon 
makl ng. 

Materials and Equipment 

Shared Decision Making Is divided 
Into an overview, eight training and 
discussion units, and an extensive 
anndtated bibliography. 

The overview gives a brief descrip- 
tion of the content area of each of the 
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eight units; It also describes the 
materials and hov^ they are best used, 
recorT¥T>ends the amount of time to 
spend Of] each unit, and provides a 
" list of questions each unit Is de- 
signed lo answer. 

these questions were raised fay 
admialstrators and teachers v;her^ 
SDM v;as being developed and field- 
tested. They help.g4jide those con- 
-^sldering training to decide which 
units suit their needs, and iTlso 
serve as a Quide for the leader 
during^ the discussion sessions. 

Unit 1. "Strengths and Limitations 
qf Shared Decision rtaking" re- 
ylev;s potential advantages and 
disadvantages of shared de- 
« cision making and presents 
several model programs for 

• shared jdecision making. 

Unit 2. '.'Personality Characteristics 
and Institutional Values" en- 
compasses psychological, behav- 
ioral, and socioVogical aspects 
^ of shared deci s-lon^nVaki ng , 

emphasizing such tbings as ^ K . 
leadership characteristics of 
''individuals, dominant cultural 

* values, human interaction com- 
ponents, as well as personal 
and group expectations. - 

Unit 3. "Decision-Making Models": 
how to build -them, what they 
should include, etc. This unft 
describes procedures used by'* 
different groups ^r reaching 
decisions and th6 techniques 
for creating Buch ,|>rocqdurqs , 
including factors tO' be con- 
side red in des Ign i ncj|\ moddl s . 

Unit "Appropriateness of Decision' 
Making" familiarizes partici- 
pants with empirical findings ' 
from the literaFure and also 
with an analysis of'schools curc- 
.rently engaged in shared de- 
cis4on making. _ The appVopriate- 
ness of decision-making 
pesponsibi 1 i ties is also exploreld. 

Unft 5. "Problem Solving and Dectslon- 
Maklna Processes" deals with group 
problem solving and comrhuntcatfOn 



skills necessary for decision- 
making' groups to function effec- 
'tlvely;^ This unit also emphasizes 
force fle-Id analysis, brainstorming, 
and a budget exercise. 
\inVt4^. "(fornmunlcatlon Systems'' em- 
^ phaslzks building tij;ust and effec- 
tive communication with people that 
are affected by group <lecl s I ons '''^^^ 
but thit are oufslde the declslcfn- 
maklng^M^aUP^- ' Several specific 
- ^^ gq est I ons are made related to 
b^ZKWIng trust. Including factors 
Influencing intergroup communlca- 
Alon, Intragroup communication, and 
quality control procedures. 

r7. -^"Accountability fbr Decision" 
helps members of a group focus 
on accountability and responsi- 
bility issues in shared decision 
making and the continuing chal- ^ 
' lenges of Insuring thqrt: decisions 
are successfully and responsibly 
Implemented', monitored, and eval- " . 
uated. Se 1 f-commi-tment processes 
are al so^^emphas I ?ed . * • 
Unl^t 8. "Organizational Output" is,. — 
primarljy concerned with evalua- 
tion of shared decision-making 
processes and procedures compare^ 
v;Ith results of more traditional, 
hierarchical, or authority-based . , 
wprk organizations. Included are 
the difficulties and complexities 
of evaluating the effectiveness ^• 
and assessment Issues. A concep- , 
* tual mode for evaluating organiza- 
tional output is briefly outlined. 

The annotated*.bibl lography of 155 
entries Includers writ'ings in psychology, 
sociology, anthropology, business 
management, lav%f, economics, and education 
that the developers considered relevant 
to shared decision making. Two aster- 
isks are placed next to t-hose ' ^ 
entries they considered bcfsic to <^n 
understanding of shared decision making, 
and one asterl sk*^ (-O next to thos'^* that 
may be of Interest. Three- as ter I sks 
(vv^*.^^;) denote a bibliographic vjorkS\ 
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How to Use 

<<? . 

The eight units of the Shared 
Peels Ion Maki ng prtjgram are not ' 
' necessarily designed as an integrated 
set, although eachfunitMs comple- 
mentary t^ the Otfiers: The develop- 
ers' had two groi/ps .in mind when 
designing SDH : , those wi'th some, ex- 
perience in shared decision making 
and those with little or' none. They 
suggest thofe in the ^second group 
b^^frv with the unit "Problem Solving 
and Decision-Making Processes." 
Those in the fi^rst group 'should review 
the overview questions to de^termine 
-^hlch unifs best suit their needs . 

.Units 1, 6^and 8 require reading 
a^papl^r and then spending between one 
anci three hours discussing it. The 
overvlw questioTis can be used by the 
leader as a^guide for these discussion 
ig^royps. 

Units 2^ 3, ^, 5, and 7 are six- 
to eight-hour workshops. Each work- 
shop I ncl udes 

^ V. Leader *s . materials and 

participant's materials. These 
are identical except that the 
* leader i,s also given a list of 
material^ 'needed during the work- 
shop, ^n agenda, the list of ^ 
, performance objecti ves (see 2 
^ bfelow) and answers to the pre- 
and posttest- The leader's' 
' ' materials require one or tw6 

hpurs reading and study time for 
-each unit. The leader ^s mater- 
ials also Include performance 
objectives and a statement of the 
skills each participant mtjst show 
to Indicate that objecti ve has 
been met. V 

2. A pre- and posttest'^. The 
leader has the answers which are 
to be given to the participants 
for the purpose of self-assessment 
and discuss ion. 

3. Reading materials., fhese are 
designed to enhance the partici- 
pants* understanding, ski 1 Is, 
interest) and knowledge related to 
shared decision making so that the 



activities and exercises follow- 
ing the reading will be of greater 
value to the participant. 

The workshop units differ in the 
types of exercises and activities they 
offer. 

Unit 2, "Personal i ty Xharacter- 
i'stics and Institutional \falues,*' asks 
the participants to 1 i s t successful , 
^ meaningful experi ences they have had 
in their lifetime and then indicate on 
a chart as well as they , can why the ex- 

terience was meani ngful , e.g. , because 
he experience involved "learn^ing new 
things." Another section of the unit 
asks them* to rate 21 sets of five 
values each (achievement, aesthetics, 
health, honesty, recognition, for 
example)^ in order of "the importance 
they place on them. They then summarize 
the results of their ratings to deter- 
mine which of the 21 values they rate 
highest. Other exercises ask the par- 
ticipants to compare *thei r own values 
and assumptions about children's learn- 
ing with those of their school. 

In Unit 3, "Decision-Making Models," 
each participant completes a paper and 
pencil exercise that requires making 
decisions in a simulated situation. 
Each participant rates a list of 15 
items such as compass, water, etc., 
according to their impbrtance on a 200-^ 
mile trek across the surface of the 
moon- The group discusses the different 
types an^ elements of decisions. Groups 
of from four to six members are formed 
and the paper and pencil exercise is 
repeated ,^ wl th .the group arriving at a 
consensiJs.' Each person can arrive at 
their error points by comparing their 
own answers with the group answers with 
the National Aeronautics and Space Admin- 
istrator's (NASA)" Answers . A form is 
'provided for this purpose. A paper 
exploring and analyzing three different 
types of decision-making models and 
discussing their advantages anj^ disad- 
vantages is provided and can be discussed, 
if desired. 

In Unit ^, "Appropriateness of 
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Decision Making,*^ the participants 
familiarize themselves with some of 
the Information currently available 
on shared decision making and discuss 
and analyze the diagnostic tests 
they have taken. Two optional phases 
of the unit follow, both of which 
require role playing. In the first, 
described as a loosening-up exerxise, 
teams are required to act out silently 
some preposterous group task, such as 
getting a 5,000 pound elephant out of 
the mud. This activity is supposed to 
demonstrate the importance of non- 
verbal communication. The second 
simulation requires the team members 
to assAime staff rojes—admi ni strator , 
department chairman, first year 
teacher--and participate in a group 
decision-making activity in acpordance 
with the interests and perspectives 
of the role they are playing. 

Unit 5, "Problem Solving and 
Decision-Making Processes," includes 
a simulation game called "The ^Budget" 
In which participants-each playthe 
role of 'a person on a planning commit- 
tee that has to decide what to do with 
an extra $10,000 in the school budget.'' 

There is an exercise in brain- 
storming requiring participants to 
think as quickly as they can of 
possible uses for two train carloads 
of ping pong bal Is cut in half. 

Unit 7, "Accountability and 
Responsibility," contains an account- 
ability instrument that requires groups 
of from five to ten participants to 
prepare an approach or system for 
improving follow through on decisic^ns 
ma^e. The group then attempts to 
arrive at a consensus for an approach 
that could be incorporated into the 
ongoing, shared decision-making pro- 
cedures at their school, if they 
exist. The training should enable 
participants to complete a plaai?ed 
design for uphol ding decis ions and 
specific plans for arrival at group 
consensus and commitment. 



Trainee Assessment 

The pre- and posttests provided 
with the five workshops are brief, 
written, true-false tests. The answers 
and a short, wrTtten discussion of them 
are given to participants, and time Is 
provided for comment. Units 1,6 and 8 
do not have testing materials. 

Product Development and Evaluation 

Shared Decision Making is one of 
a series of training modules developed 
by the National Cluster Coordination 
Center (NCCC) . The others are Func- 
tional Task Analysis , Communication 
Information Flow , System Renewal , 
Organizational Crisis Intervention , 
and Performance Objectives , all of 
which were to be available as of Decembe 
1974 from ACSA. ' ^ ^ 

The NCCC was organized to develop 
these modules under the direction of 
the School Personnel Utilization program 
(SPU) of the Educational Personnel 
Development Act (EDPA) . SPU determined ^ 
through surveys and studi"es that shared 
decision making is a skill essential to 
change. Over a three-year period, the 
EPDA project attempted to implement 
shared decision-making models in schools 
and discovered that teachers and admin- 
istrators were not prepared fpr this 
type of collaborative operat i on . , The 
NCCC stepped in and began reviewing 
over 300 published articles, books, 
training manuals, and research projects, 
in addition to surveying 11 EPDA' project 
schools that were already involved in 
some form of shared decision making. 
The NCCC identified eight areas where 
they felt training materials vyere not 
available but needed. These eight areas 
are represented by the eight utiits in SDM 



Actual development began in July 
1972 and was completed in September^973. 
Each^unjt was conceived and developed 
independently but with the intention 
that irb* part of the whole program. 
Developers i ntended that the user/client 



68 

7d' 



Identify the cdmponents or elements 
unique to his or her situation by 
reviewing the overview to SDM and 
using only the relevant training 
sections, \ 

SDM was field-tested in the 
eleven EPDA project schools with 
approximately 200 nonproject teacher 
and administrator volunteers. All 
field test participants were con- 
sidered to be the SDM target audience. 
Field test resu 1 ts were pK)s i t i ve and 
indicated th^t the module contents 
and activities helped teachers and 
administrators in developing a 
collaborative, participatory, 
decision-making, system within the' 
school setting. 

Costs 

Shared Decision Maki ng comes 
slickly wrapped and three-hole punched 
in one package that includes all 
leader materials and participant 
materials. Those interested can 
order one set for each participant 
and remove leader materials from the 
set given to participants, or order 
one set for the leader and* dupl i cate 
the necessary forms, for participants. 
The cost is $12 for members of ACS A 
and $15 for nonmembers. 

Comment 



Shared Decision Making thoroughly 
treats the^ problems and possibilities 
of shared declTion making. It pro- 
yides much material for thought and' 
discussion, makes (recommendations, 
and reports f i ndirt^s><« but draws no 
absolute conclusions. It does not, 
for example, conclude that shared 
^decision making is superior to indi- 
vidual decision making, but instead 
asks participants to weigh ^he^rgu- 
ments presented and come t^p^nef r own 
conclusions. It should bcfnoted that 
not all Units present their material 
with equal clfiH4-ty., 

SDM appears td be valuable both 



for group and individual training, 
Its low cost puts it within reach of 
individual purchasers. An indlvidua*l 
interested In shared decision making 
could profit from simply reading the 
materials and taking the pre- and 
post tests,. 
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TRAINING RESOURCES 



This sect\ojf\ describes resources 
available foj>dse in training programs 
in instructional planning^and rtianage- 
'•ment. The resources by^hemseNes 
are not complete or comprjj|feiis i ve 
enough to conduct a full-fledged 
training program; instead, they are 
tools to support or supplement train- 
ing. These resources include mostly 
reference^ books, handbooks, textbooks, 
monographs, and case studies. The 
brief description of each resource 
indicates its author, publisher, 
length, cost, content area, and tar- 
get audience. Possible training 
applications are also suggested. 

The resources are organized 
topically under five major areas: 
(l) purposing, "n^. , analyzing prob- 
lems and needs, identifying goals, 
specifying instructional objectives; 

> 



(2) program planning, i.e^, analyzing { 
and selecting experiences and resources 
to be included in a program and deter- 
minttig their organization ; (3) program 
development, i .e .-*J^dopt i ng^ adapting, 
or inventing new methods of instruc- 
tion; (4) progijara evaluation, I .e. , 
identifying nee3s and selecting pro- 
cedures for focinative and summative 
evaluation, determining adequacy of 
program design and implementation, 
assessing adequacy of the program, 
achievement of program goals, and cost, 
and determfning modifications needed; 
and (5) organization for problem 
solving, i.e., arranging for and facil- 
itating instructional planning and man- 
agement decisions. The listing of 
resources is not exhaustive; It should, 
however, alert you to a variety of 
*materi als useful for planning and 
conducting training programs. 
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purposing 



Gottesfeld, Harry. Educational Values Assessmerit (EVA) Questionnaire. 
New York: Behavioral Publications, 1973. 24 pages. $2.95 



The EVA quqstionnai re is an instrument for discoveri n/what is educationally 
important to i ncU vi dua I s , schools, and community groups, /it helps to identify 
differences i-n v^ues and sources of disagreement between groups. It may be 
administered to/^rents from various ethnic and socioeconomic groups, community 
leader^, and teichers and administrators from various backgrounds, in different 
types of sch0ots\ The information gathered could be useful to educational and 
community planners, schoo^-l administrators, and researchers. Participants rate 
the importance of 80 items in the following categories: community involvement; 
strictness standards; professional specialists; innovation; health, recreation, 
and practical training; economic considerations; parent education; and special 
handling of difficult children. Demographic characteristics such as the sex, 
age, ethnic group, and education of the participants are also noted. An EVA 
questionnaire, scoring sheet, and manual are the basic materials in the setT The 
manual contains instructions for administering the EVA and describes its devel- 
opment and field testing. Additional questionnai res^, in sets of 25, cost $5.^5; 
50 scoring sheets cost $5.00.. Jhe questionnaires maVN^be hand scored, although 
the^developer will score them electronically. Interpretation of the meaning of 
the scores should be done by someone familiar with both attitudinal scales and 
the particular participant group and school in question. 

V r • , ' ^ 

. V V V 

Hoepfner^ Ralph; Bradley, Paul A.; andboherty, William J- National Priorities 
for Elementary Education. Los AngeTi&4: Center for the Study of Evaluation, 
UCLA Graduate School of Education, 1973. 72 pages. $3.50. 



This monograph, the second in a series, addresses the topics of educational 
needs assessment, different methods of determining needs, and the developers* 
particular method. The national priorities to\;/hich the title refers are those 
that surfaced in response to a nation-wide survey of elementary school goals. 
Results of the field testing of the CSE Elementary School Evaluation Kit: Needs 
Assessment (page'^59) are reported.'^ The sample included a lacge number of schools 
differing in geographic location, population density, size, socioeconomic 
status, and raci a I -ethn i c composition. In each locale, teachers, principals, and 
parents rated the importance of 106 s tudent-outeome goa I s . Numerous tables com- 
pare the responses of the different groups. The content of the monograph's four 
chapters is as follows: 

Chapter One examines educational needs assessment and describes the model 
used by the Ce^iter for the Study of Evaluation (CSE) . Chapter Two dis- 
cusses the rationale and development of the CSE goal taxonomy. It also 
details the composition of the sample and describes the goal-rating pro- 
cedure. Chapter Three summarizes the findings of the survey and reviews 
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similarities and differences In ratings by the different groups 
Chapter Four considers the implications of the methodology and findings 
The monpgraph cone udes with suggestions for use of the Information and 
precautions about interpreting and applying the findings. 

The monograph would be ofVs^test Interest to elementary school principals, 
evaluators, and researchers. *^ i.iv-ipaia, 
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fn!*Jl!n!]°"^^ Objectives Exchange. Measurable ObjeaHvee Collections. 
Los Angeles: Instructional Objectives Exchange, n.d. $8.00 each. 

The Instructional Objectives Exchange (JOX) has compiled approximately 50 
collections of c"ogn, t . ve and affective instructional objectives ^ Each collection 
focuses on one instructional area, e.g.. Reading Comprehension Sl<ills. K-12 
-Ob iBcH !^ "r - ^""^ correct responses accompany each objective. For example. 

r ir r" ^ ="PP'y ^" antonym" is accompanied by 

.rZlrl in' ^".^"'""V" of the following words"; the word and 

answers follow. Objectives are arranged In categories: "Major category Com- 
a's"" 'x"9"'bo"^"'^;"^"" Z'. '."^"^-"'^?-y= Antonyms." Each^coHe^^L'is bold 
\Lr. n ; '"/^^^tion to objectives for matheaiatics and language arts, 

.Jhnnl ! l°ns for American history, music, self-concept, attitudes towa>d 

school anthropology, sociology, early childhood education, knoviledge of and 
attitudes toward drug use 1 i fe sciences, fo re i:g|i. languages . business education. 
home.economics - vocationa arts, and others. The developers suggest several 
lul llZ^V [u Teachers can supplement their own objectives 

with those from the collections, can adapt selected objectives to their own 
teaching situation, or can use them as models for devising their own objectives. 
Students could be involved in planning their own learning by choosing objectives 
themselves Students, community members, and professional staff might assess . 
the school s needs by ranking the objectives. Procedures for such a needs 
assessment would have to be worked out by the school because none are given. 
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Klein, Stephen P.; Hoepfner, Ralph; Bradley, Paul A.; Woolley, Dale; 
Dyer. James S.; and Strickland. Guy P. Procedures for Needs-Assessment Eval- 
uation: A Sympostm. Los Angeles: Center for the Study of Evaluation, UCLA 
Graduate School of Education. 1971. ED 055 111. 52 pages. Microfiche , 
$.65; hardcover, $3.29. 



This report on a symposium at the American Educational Research Association 
Annual Convention covers four topics. 

(1) "Choosing Needs for Needs Assessment." by Stephen P. Klein, presents 
a new technique for conducting the initial steps of a needs 
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assessment. In essence, this technique is to have experts 
who have the time, knowledge, and resources construct compre- 
hensive sets of goals and objectives. The experts provide a cata- 
logue from which decision makers can select appropri ate goal s for 
their specific situation, rather than having to adopt a total set 
of goals or objectives prescribing what a school or program 
should do. 

(2) "Selecting Tests to Assess the Needs," by Ralph Hoepfner, is a 
summary of the MEAN method of selecting tests. 

(3) "Making Better Decisions on Assessed Ne^ids : Differentiated School 
Norms," by Paul A. Bradley and Dale Wool ley, is. concerned with ways 
in which data obtained from the assessment of s tude'nf performance can 
be improved so that it is more useful. 

(A) "Allocating Resources by Subject Area," by James S. Dyer and Guy P. 

Strickland, describes a procedure des i gned to assist elementary school 
principals in the process of selecting educational subject areas which 
should command their attention, resources, or support. A model is 
presented which produces an Index number for each subject area. This 
index number represents the expected "value" which will accrue to the 
school from the adoption of an instructional program appropriate for 
strengthening the specific subject area. 

This report also contains a list of references and an appendix which lists the 
goals of elementary school education from the CSE Elementary School Evaluation 
Ki t , described on pages 59"6l . 
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Nummedal , Susan G. CSE Preeahool/Kindergarten Hierarahioal Objeotivea Charto. 
Los Angeles: Center for the Study of Evaluation, UCLA Graduate School of 
Education, 1971 . $10.00. 



The Preschool /Ki ndergarten Hierarchical Objectives Charts are comprehensive 
exhaustive, logical hierarchies of all goals and objectives identified for pre- 
school and kindergarten programs--Speci f i ca I ly , programs for chi Idren between 30 
and 72 months of age. Because o.f the movement in public education toward school- 
ing for four- and five-year olds and the demands for accountability, there is a 
tremendous burden on those responsible for the education of young children to be 
explicit about the goals and objectives of their programs. These charts were 
designed to aid early childhood educators in the Jobs of curriculum planning and 
assessment and to help them meet the demands of accountability by acquainting 
them with a range of objectives that might be adopted in an early childhood 
education program. The 21 major goaP categories are development of personality; 
social development; development of motivation for learning; development of 
aesthetic appreciation; arts and crafts; cognitive functioning; creativity; 
memory; physical coordination; foreign language; function and structure of the 
human body; health; mathematics; music; oral language skills; religion; readiness 
skills; social studies; science; safety; reading; and writing. To Identify these 
goals, an exhaustive search of program and research literature was conducted, as 
well as extensive Interviewing of educators and early childhood specialists. 
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The hierarchies for each goal category represent the broad range of possible 
objectives for preschool and kindergarten programs and are designed to be useful 
for students of all ability levels withip the age group of-concern. Because of 
this broad raoge, the charts are strictly a "presentation of the possibilities;" 
It Is still up to the educator .to decide the priority order of the objectives 
selected, determine which are applicable to the skill level of the students, and 
turn them into specific, measurable behaviors. These charts are an outgrowth of 
the taxonomy, of goals established for the CSE-ECRC Preschool/Kindergarten Test 
Evaluations book (see page 8l ) . The two products may be useful to those con- 
cerned wi;th assessment of program outcomes. After objectives have been identi- 
fied in one of the goal areas of the charts, the test evaluations book can then 
be consulted to identify evaluation instruments available for those objectives. 
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Maguire, Louis M. ObsewationQ and Analysis of the Literature on Change, 
Philadelphia: Research for Better Schools, 1970, 42 pages, $3.40. 



This two-part report reviews various materials written on change, particu- 
larly change in educational settings, and analyzes the subject and the writings. 
Part I, "Observations on the Literature for Practicing School Administrators,'' 
deals with the value of the literature to the practitioner. The author's ob- 
servations are presented under the headings of (1) conceptual confusion; (2) goals 
and objectives; (3) statement of problems; (k) democracy and planned change; 

(5) the school district as a target or initiator of planned change; (6j internal 
and external linkage; (7) change capacity; (8) maintenance or improvem^t; 

(9) chanjB«-mcKie)s; (lO) phases of change; (11) roles in change; (12)/cri\is as a 
stimulus to change; and (13) lack of training. Part II, "Review of/ Li teriture ," 
discusses varipus viewpoints on change under the following headings/; 0)3*^^'"^" 
tions and types of change; (2) change models; (3) strategies and tiol^ULLo^ues ; 
{k) people involved in change; (5) sources of and barriers to change; and 

(6) research studies of the change process. The author contends that for the 
most part, the literature on change provides little that js easily and immediately 
usable by the school administrator because it tends to portray change as a novel 
event, while the administrator knows that it is occurring all the time. It should 
be noted, however, that the author*s most recent example of the literature on 
change was published in December I968. He commends the potential of a number of- 
the models, some of which were in thetr earlier stages when he reviewed them and 
have been developed further since thatrtime. 
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Maguire, Louis M. ; Temkin, Sanford; and Cummings, C. Peter. An Annotated 
Hbliorjraphy on AcMniotering for Oicmge. Philadelphia: Research for Better 
Schools, 1971. 333 pages. $3.40. 



This annotated bibl I ography is Intended to reflect the direction of the 
Administer ing for Change Program ( ACP) at Research for Better Schools. The vol- 
ume is^divided into six sections: an introduction; four sections which reflect 
the four components of the Administering for Change Program: "Knowledge Base,"- 
Orqanlzlng for Change," "Planning for Change," and "Managing for- Change"; and a 
last section of bibliographical and anno4:ated bibliographical entries. The 
"Knowledge Base" component surveys, collects, analyzes, evaluates, synthesizes, 
and reports field data and literature information to assist. In the cbnceptuali- 
-zation of change processes and to support other program Components. The 
"Organrizlng for Change," "Planning for Chang^," and "Managing for Change" com- 
ponents, supported by the "Knowledge Base , ''■'•are directed toward developing a 
system of training and reference materials which will enable a school district 
to identify and implement needed changes. Each of the components Is best viewed 
as producing an essential element of change. Each section Includes the following 
specific subjects: 

Section Two, "Overview of Change Literature," Includes definitions, models, 
strategies, techniques, roles, and barriers to change. Section Three, 
"Organizing for Change," covers institutional norms, goals, structure, de- 
velopment, environment, and processes,- Section Four, "Planning for Change," 
Is divided into two parts. The first, "System Wide Planning," Includes 
PPBS; cost-effectiveness; Input-output analysis; and cost/benefit analysis. 
The second part, "Comprehensive Planning Process Information »" covers 
pupil needs; wei gh Fng goals , objectives, and outcomes; performance and 
educational process criteria; cost and accounting; and Implementation. 
Section Five, "Managing for Change," is also divided Into two parts. 
The first, "Educational Project Management," covers alj types of management 
skills such as managemen t, references ; Introduction to project management; 
management control; and problems in project management. The second, 
"Educational Problem Solving," includes decision making; administrative 
tools, techniques, and skills; creativity; research studies; and theory 
and methods . 
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Marian, Michael D. Alternative Futures for Learning i ^An Annotated Bibliography 
of Trends y ForeaaQto^ and ProposalG. Syracuse University: Educational Policy 
Research Center, 1971. 223. pages. $5.00. 



Although the author calls this an "Incomplete bibliography" and "an attempt 
to sketch out "the range of 1 1 t<,i^rature that Is relevant to educational policy- 
making," It proves to be quite comprehensive. The bibliography concentrates on 
"trends, forecasts, and proposals--or documents on changes that are taking place, 
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future states of affairs that may occur, or recommended states of affairs that 
ought to occur." It presents works from many camps. "Establishment or anti- 
es tablishment , technocrat or humanist, radical or conservative," presenting an 
array of ideas about "who should learn what and how." In addition to those 
dealing specifically with learning", a large number of general background works 
are also described. All of. these either make reference to or have i rnp 1 i cat i ons 
for education. The 936 items are arranged in six major categories: general 
futures literature; elementary and secondary education; higher education; other 
educating insti tufions ; planning and plans, and a miscellany section which in- 
cludes pre-1960 forecasts, new and relevant periodicals and bibliographies. 
Some of the more interesting of the kO subcategories in these sections are 
socia^l change and social goals; impacts of technology; youth and youth culture; 
the knowledge explosion; state, national, and g 1 oba 1 perspect i ves ; curriculum; 
urban schools; graduate education; preschools; adult and continuing education; 
electronic media; forecasting methodology; planning and policy making; and 
planning for change In education. In his foreword, the author makes certain 
observations on the literature. He sees as a major theme the necessity of re- 
placing' closed teaching systems with open learning systems. He also discusses 
the neglect of adult and continuing education in the literature, notes that 
writers are becoming more prescriptive than descriptive, remarks on the relation- 
ship between education futures literature and ger\eral futures literature, and 
makes recommendations for further work. Two other bibliographies by thesame 
author are available. Essential Reading for the Future of Education: A Selected 
and Critically Annota ted~Bib 1 i ograp hy (a shorter version of Alternative FutureT 
for Learning) provides an introduction to and overview of the subject. It is 
available at $1.50 from the publisher. Alternative Futures for Mankind: An 
Annotated Bibliography of Societal Trends, Forecasts and Proposals looks at 
futures literature "in all categories: general overviews, science and technology, 
society, polity, economy, environment, learning, methodology, pre-WorldWar II 
forecasts, selected Utopian and science-fiction writings, and forecasting..." 
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School Management Institute. PPHO (Planning, Programming, Budgeting Syatern): 
For People Who Don't Underotand PPBS. Worthington, OhioV School Management 
Institute, 1971. 23 pages. $1.75. 



This compact, highly readable booklet is an introduction to PPBS . A rational 
for adopt ion--the importance of accountability and the emphasis in J>PBS on results 
and measures for obtaining them--precedes a description of the PPBS cycle. A 
chart shows the sequence of steps in the cycle and the text di scusses them. In 
PPBS the whole staff, students, and community participate, whereas in traditional 
budget planning the school business official works alone. PPBS , the booklet ex- 
plains, is not primarily concerned w i th finances, even though budgeting Is in- 
volved. Reviewing costs, benefits, and resources form only one aspect of a cycle 
that also includes needs assessment, goal setting, program selection, and trial 
and evaluation. Besides explaining PPBS , the booklet dtscusses a few of its ad- 
vantages and some of its requi K^judnts . For e;^ample, the time required for full- 
scale operation of the system can be considerable; also-, it is essential that 
various groups be included in a number of the planning phases. The target 
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audience, d« i de from, "people who don't understand FPBS ," is not specific 
Likely usersr. however, would be district superintendents and school board mem^ 
bers considering PPBS and needing a direct, concise description of it Principals 
teachers, cur r i cul um:speci a I i s ts , and others concerned wi th instructional planning' 
tn^y also want to consult this booklet. Illustrations from the experiences of 
other districts that have used PPBS make the booklet more credi-b^e-a'nd interesting 
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Temkin, Sanford. An bvaluatton of Comprehensive Planning Literature with an 
Annat.at*\l Bibltography. Philadelphia: Research for Better Schools 1970 
82 pages, $3.00 



This book IS a comprehensive listing and an evaluation of the planning liter- 
ature designed to help schools plan to meet goals and objectives. In this context, 
however, "comprehensive" does not mean completeness of information but ra-ther 
quality of information. The section preceding the bibliography discusses and 
evaluates many ideas and methods that are to be found in the literature on compre- 
hensive planning. The second section is an annotated bibliography, which is 
divided into five sections: (I) "Planning Process Information"; (2) "System 
Wide Planning Methods"; (3) "System Characteristics and Qualities"; (^4) "Politics, 
Community, Implementation and Communication"; and (5) "Bibliography." An author 
and subject index follows the bibliography. 
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Heinich, Robert. Technology and the Management of Inatruation. Washington C 
National Education Association, 1970. 198 pages. $6.50. 



"The paradigm of instructional management generated by comprehensive tech- 
nologies of instruction is fundamentally different from the paradigm— and support- 
ing s(jperstructure— that has evolved ever since the person in face-to-face contact 
with students was vested with institutional authori ty . . . I t is my position in the 
monograph that Thomas S. Kuhn's model of scien t i f i c revol ut ions is applicable to 
fields other than science. I am even more firmly convinced today that because of 
technology, anomalous situations in education exist and will have to be resolved 
through adoption of new paradigms; management of comprehensive technologies cannot 
be extensions of traditional relationships any more than the management of' the 
industrial revolution could be an extension of the guild system."' With these 




'copyright (c) I97O Robert Heinich and the Departmlpnt of Audiovisual Instruction, Inc. 
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opening sXatements, H4inich begins his analysis of two paradigms of ins'^truct ional 
management: the traditional paradigm and a new paradigm ref lect 1 ng the topact of 
Instructional technology. Helnich defines Instructional technology as? th^ 
speciality within education whose concern centers on the application of tech- 
nology and technological concepts to instruction. General systems theory is 
among those technologlcaj concepts with Implications for education. Among the 
factors contributing to the need for a more comprehens I ve systems model for In- 
structional management are the following: the evolution of media from Instruc- 
tional aids to Instructional m^ans In their own right; the adoption of media by 
the disciplines; and the demands for inexpensive, highly effective modes o^" in- 
struction. Helnich contends that *'a fundamental cause of system redesign Is the 
development of sufficient energy within a subsystem or subsystems, to force a new 
analysis--synthesls sequence, resulting in a change In the paradigm, or conceptual 
framework of the system. Sufficient energy has been generated in the media sub- 
systems of instructlonVto force a redesign of the paradigm of instructional man- 
-agement."2 Helnich , .develops his model systematically and effectively in the six 
chapters of the monograph. In addition to pointing out the need'^fpr a new par- 

, adigm, he discusses the reasons for Its opposition by defenders of the traditional 
paradigm. Chapter Two contains an interesting discussion on creativity and the 
generation of new theories in fields other than education. Chapter Four, which 
considers the influence of 1 nformation theory on various learning theories, tends 
to be technical, although it should be clear to persons familiar with the^subject, 

Overall, this is a valuable item for administrators, media specialists, currlc^^um 
planners, and r,esearchers . 
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Klausmeier, Herbert J.: Quilling, Mary R.; Sorenson, Juanita S.; Way, Russel S.; 
and Glasrud, George R. individually Guided Education and the Multiunit Elementary 
School: Guidelines for Implementation. Madison: Wisconsin Research and Devel- 
opment Center for Cognitive Learning, 1971. 132 pages. $2.00. 



This book discusses Individually Guided Education ( IGE) , a comprehensive 
system for 1 nd 1 v idual 1 Zvl ng elementary school 1 ns tr^iJCt^or}^, and the multiunit 
elementary school model, the organ I za 1 1 ona 1 -adml n I ^t ra 1 1 ve component of the IGE 
system. Other components Include a model for Instructional program planning for 
the Individual student; a model for developing measurement tools and evaluation 
procedures; curriculum materials (including Instructional object 1 ves , cr i ter Ion- 
referenced tests, and observatl on scales); a home-school communications programj 
and a model for supportive relations between school st^ff, school system central 
offices, and state education agencies. The organizational component consists of 
a three-level, hierarchical stafffng arrangement. At th€f classroom level, non- 
grade instructional end research (l & R) units replace traditional age-group 
classes. Each unit has a unit leader, two or three staff \eachers , one first- 
year teacher, one teacher Intern, one aide, one Instructional secretary, and 
100-150 b'tudents. Unit level functions include planning instruction for Individual 
students and teaching Individual students, small groups, and large groups. At the 

1 ' 
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building level, the principal and unit leaders work together to set curriculum 
objectives, develop 1 ns true 1 1 ona 1 prog rams for the school, and coordinate activi- 
ties for all the units. This group, called the Instructional improvement 
committee.. (IIC). also interprets and implements systemwide and statewide policies 
that have bearing on the school prdgram. At the third organizational level 
systemwide policv^ committee (SPC) makes the major decisions concerning the^ 
functions of each raultiunit school in the district. Other responsibilities are 
the recruitment and inservice training^of personnel, providing instructional 
materials, and disseminating informatioh within the district and to the community. 
The superintendent, central of f ice staff, representative principals, unit leaders, 
and tea^chprs make up the committee. Inservice staff training in the procedures 
^or adopting the model and for planning instructional programs is also part of the 
design. The following training schedule, is given: (l) a two-day overview confer- 
ence for administrators and central office personnel; (2) a three-day workshop 
for principals and unit leaders; (3) 16 hours (four i^hours , once a month) oi pre- 
installation inservice training for the entire staf^; {k) a three-to-five day 
workshop for the entire staff before the beginning of school and four half-day 
sessions for the entire staff, distributed throughout the first year. Full in- 
formation on costs and requirements for adopting the model Is provided in the book. 



program evaluation 

EPIC Diversified Systems. I'vogram Evaluation Packet. Tucson: Educational 
Innovators Press, 1970-73. $25.00. 



The Program Evaluation Packet js a series of twelve booklets on th-e various 
aspects of program evaluation. The first booklet In the series, A Scheme for 
Evaluatl.on and An Organizational Structure of Variables . Introduces the evaluation 
model used throughout. It briefly defines evaluation and describes a four-phase 
scheme for evaluation. The tasks undertaken In these four phases--p 1 ann I ng , 
implementation, product, and recycl I ng--are discussed at greater length in later 
booklets. In the second part of this booklet, an "organizational sti^ucture of 
variables'^ Is offered as a systematic procedure for identifying and describing 
the variables I nf 1 uenc I ng ^an Instructional program. Th I s , s tructur^ reappears as^ 
an aid to evaluation In a number of the booklets. Titles of the other |[^oklets are 

Developing and Writing Performance Objectives 



Coding and Selecting Test Items 
Proposal Gu Idel I nes / 
Developing and Writing Process Objectives J . ^ 

A Format for Monitoring the Teach I ng-'Learn I ng Process 
Affective Measures for Educational Evaluation 
Hierarchy for Goals and Objectives 
Educational Program Audit ^ \ 

Developing the Accountability File 

A Pltihnlng^ Monltorlngi and Evaluation System for Career Education Programs 
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■■^ Booklets may be purchased In the packet or separately. They may also be comlj^lned 
with others from the publishers' series of 29'bQoklets to form packets on l;^eacher"' 
evaluation, accountability, management, program development, an«^ur r I cu 1 urn- devel - 
opment. Topics of the individual booklets i ncl ude needs' assessment , case studies, 
performance-based Instruction, I nteract Ion ana lys 1 s , ^ and others^ Possible users 
of the materials are' local and state level adml n1 s t ra tors consu 1 tan'ts , project 
directors and staff, evaluators, research and development professlQnal s , and class- 
room teaqhers. The booklets are prlmarliy Intended to provide basic background 
information. Persons wl th cons Iderable experience and training In^ the subjects 
could use the packets to conduct training v^orkshops. 
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Hively, Wells; Maxwell . Gj^aham; Rabehl . George; Serrsion, Donald; and Lundin, 
Stephen. Domains-Re fr,rent!^ed CuvyHcuLum KDaluation: A (\nu^ r^twlij from the 
MIMEMAST PvojecJt. Los Angeles:^ Center for the Study of Evailuation, UCLA 
Graduate School of Education, 1973. 98 pages* $3.50. 



tech- 
Eval- 
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The first In a projected series on evaluation, this monog^f^ph discusses 
TOma In-referenced tests, their design and application, and the context in which 
they were first developed. It Is^ both a case history and a technical handbook 
containing detailed procedural guidelines for constructing tests. Chapter One, 
"The MINNEMAST Curriculum Project," describes, the Minnesota Mathematics and 
Science' Teaching Project, Its aims, development of materials, 1 nstruct fonal 
nlques, and Initial evaluation. Chapter Two, "Domain-Referenced Curriculum 
uatlon In the MINNEMAST Project." outlines the rationale of the measurement 
tem. This section explains the procedures employed i n -^pvel op I ng test-Item 
domains, Composing test-Item forms, experimental design, test construction And 
•edmlnlstratlon, and -In ferpret I ng the res-ults. Chapter Three, "Considerations In 
the Design of Future Systems of, Cu r r 1 cu 1 um Construction and Eva lua^t ion ," analyzes 
practical problems In currlculuijn development (such as organizational factors 
affecting, evaluation effectiveness) and recommend strategies for overc^ikhg' d i f- 
flcultles. A set of appendices provides numerous examples of^ te^t 1 temsand sec- 
tions of- reports. The monograph Is Intended primarily for researchers, curriculum, 
developers, and project evaluqtors. Chapter Three shoald be of special interest' 
to persons concerned with Instructional development. 
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Hoepfner, Ralph; Stern, Car'olyri; and Nummedal » Su'san G.. eds. 
Kindrvgavten Teat EvaLuatiom. Los Angeles: Center for the 
and the Early Childhood Research Center, UCLA Graduate School 
54 pages. $5.00. 
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( \ ' if*/- E( 'R( ' Pve^hool 
of Evaluation 
Education, 1971. 




This book Is intended for d I rec tors , pr i nc 1 pal s , and counselors concerned wl th 
preschool and kindergarten children; educational researchers and test publishers ' 
may also find It useful. CSE-ECRC Preschool /Kl rydergar ten Test Evaluations begins 
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wtLn a concise and Interesting introduction, "Accountability in Early Childhood 
Education/' which includes a brief discussion of ^evaluation in Head Start pro- 
grams. A professional evaluation team rated 120 tests (including 63t) subjtests 
with separately normetf scores) fthat measured atta^i nment of either preschool 
(30-59 months) or kindergarten (60-72 months) goals. Separate tests for pre- 
school and kindergarten children have been rated. The rating process used was 
the MEAN system (for'^en explanation jof this system, see CSE Elementary School 
Te^t -Evaluations, this pdge) . According to the editors, a wide spectrum^? 
practitioners, including teachers, supervisors,, and early childhood specialists, 
were surveyed and an exhaustive search of both the. program snd research litera-' 
ture conducted to select goals for preschool and kindergarten education. ^The 
goals were then translated Sinto operational definitions. Goals were classified 
according to the followijjfr^ affective ddmaln— personal ity, social skills, mo- 
tivation for learning, aesthetic appreciation; intellectual domain-^cogni ti ve 
tunctioning, creativi ty, memory} p^sychomotor Uoma i n— phys i cal cooiMi nation ; 
subject achievement domain— art's and crafts, foreign languages, function and 
structure of human body, health, math, music, oraTlangu^ge skills, readiness 
skills, reading and writing, religion, safety', science, social studies. The 
book inclu(jes an index of goals, a test name index, and an index of publishers 
and addresses. For a description of' how to use, the book, see CSE Secondary 
School Test Evaluations , page 83. . ^ : ; — - 
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Hoepfner, Ralph; Strickland, Guy; Stangel , Gretchen; Jansen, Patrice; and 
Ratal ino, Mari'&nne. CSE Elementaxnj School Test Evaluations. Less Angeles: 
Center for the Study of Evaluation, UCLA Graduate School of Education, 1970 
146 pages. $5.00. >^ , , 



This bool^ contains evaluations of vi rtya\;ly^very output measure putl ished ■ 
that i.s generally available to educators and researchers for use in testi,ng 
elementary schools, programs, and students. Over a fhousand output measures were 
evaluated by measurement experts an^ducators using the MEAN . Tes t .Eva 1 uat ion 
Rorm. MEAN is aji acronym for the criteria used to evaluate the tests: Measure- 
ment Validity, Examine^ Appropriateness, Administr^i ve Usability and No7med / 
Techn4cal Excellence. Each of these main ca teg'Sj: i es , i s bwrken down into sub- 
categories. For example, vMeasiurement Val id ity contacts ((^wo subcategories: * 
(1) Content and Construct, and (2) Concurrent and Fred i«<ft i ve. The introduction 
to the book defines and explains the MEAN ew^iluation method. All the*evaluati ve 
data gathered for each output measure Is errteredAan the form i/r-the appropriate 
subcategory. A poor, fair, or good rating Is assigned to e^€h of the ^ 
categories, a rating^hat has been derived from summarizing the subcaj^ergori es . \ 
For example, the Calfforn la' Language lest in spell ing Tor the th I rd /grade i s rated 
poor in Measurement Validity, fair In ExamI nee Appropri ateness , good. in Adminis- 
trative UsabI 1 I ty, and poor In Nopmed Technical Excellence, The results of .the 
evaluation bf each output measure <^ere largely determined by the purpose that 
particular measure was to serve. «\.Specif leal ly, each measure was classified as to 
its educational goal, and then its effectiveness In assessing ach I evement, I f that 
goal was evalHiated. - The^measures were judged on their appropriateness to'school 
situations, not to s-lini^al or research 'p rob I ems. The judgments arrived at shou^ 



be u sefu l t o te achers-, administrators, and educational evaluatoVs; they are- 




likely to be less useful for test selection problems, for basic research into' 
Individual differences, or for instructional technologies, although they may 
still be useful-as guides in such "p'robi em. areas. / 
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Hoepfner, Rajph, et al., eds. CS^ Secondary School Test Evaluations: Grades 
7 and 8; 9 and 10; 11 and 12. 3 vols. Los Angeles: Center for the Study of 
Evaluation, UCLA Graduate School of Education, 1974. Grades 7 and 8: 225 pages; 
9 and 10: 229 pages; 11 andl2: 339 pages. 3 vol. set: $22.00. 



The three volume set of CSE Secondary School Test Evaluations ( Grades 7 and - 
8^; 9 and 10 ; 11 and 12 ) are aimed at serving the^va 1 uat ion functions and needs 
of teachers , administrators, curriculum developers, special project staff, and 
members of state and federal education agencies. After conducting a needs 
assessment and determining that it would be necessary or desirable to assess 

^students' standing on sdhie goal, a user may consult the Introduction to one of 
the three volumes (corretepond i ng to the appropriate grade level) to find the goal 
statement most closely matching his or her need. Then, he or she consults the 
index of goals and finds the evaluation entries for all the tests published and 
available that can, be construed to measure that goal. Undfr the individual 
entries, the user will find the prjecise name of each instrument; a code .Lndncatlng 
its publisher; and ratings on 39 eva 1 uat i on. cr i ter i a . These ratings are also 
summarized into a MEAN score (for an explanation ^f this scoring process, ^ee 
CSE Elementary School Test Evaluations , page 82). By banning either the indi- 
vidual, or summary ratings, the user can select one or mor'e tests to measure 
specific goals. Next, the user would consult thfej/fndex of tests to see if the 

^ evaluation (s) chosen are single unit(s) or subtests of larger batteries. In the 
latter cajse, the user will have to decide whether the addit^ional information 
tested will be useful and rrfake tradeoff s l(o obtain the maximum useful information 
at reasonable cost. It is usually possible to obtai n' a specimen of a test, at 
nominal cost, from its publisher. Each of the three volumes in the" series has an 
index of goal categories and test names,' te^t publisher's addresses, and a list 
of curricular resources. The goals have been arranged under 16» major discipline 
areas/cover i ng most of the outcomes expected of secandary^^-s^udents throughout the 
country: arts and crafts; driver and trjaffic safety; English language art;^; for^ 
eign languages; health and safety* home economics; industrial arts; intellectual 
^'kill^; mathem^ics; mus i c^^pbrsonal i ty characteristics; philosophy and relig'ion; 
physical education; science;"" social studies; and vocational/career education; A 
professional evaluation tearp, wh6"se motto is "goal first, measure second," rated 
all of the secondary schoo^l^^^sts Histedo ^ 0^ 
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^lells , Rdn d a lT— U — Innovationo Eva luaH ^- Guide . 



Willie 

Columbus: The Center for Vocational and Technical Education, Ohiio State^ 
University, 1972. 14- pages. Available on request. 



V 



The Inndvations Evaluation Guide 



\ 



I.S 



9 tool for assessing the merits of i nnoVa-' 

tions that are either currently being tried or being' cons i dered-for aOoption. The 
guide allows users to identify the dist irvguishing features of innovations, then use 
this information to compare and select according to their n*eds . Factors such as 
pupil growth, efficiency and effectiveness of program" operations , benefits to school 
and community, amotlnt and source of mone/ needed , time requi remeots ,' personnel needs 
and a number^of other factors are considered.. Questions under kl headings assist 
the user in deciding what assets and requirements an Innovation shoiTld have. .Space 
IS providedjn the guide for recording responses. A checklist of characteristics 
and a worksheet for major costs are other. aids. Persons who evaluate educational 
innovations could find the guide helpful. Possible users are administrators 
teachers project directors, pilot program supe rvi sors , state. educat i on department 
personnel, teacher educators, research and development personnel, and change agents. 
T*e guide is the result of a 1 i terature review; i nterviews with superintendents; 
.expert review and pi lotvtest of a protckype; field tests with teachers, administra- 
tors, state supervisors, Sind local directors of innovative programs; and final re- 
vision. The^Classification and Evaluation of innovations for Vocat'ional and 
Technical Education," ^ report on the development and testing of the guide, is 
available on request from the developer. Altftiough the guide was developed by 
specialists in vocational and technical education, the items are appli-cable to all 
typesof innovations. The only J tern specifically ;^ocational is "Entry and Advance- 
ment in an Occupation'' and it may interest nonvocational edtfc^tors also. 
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Instructional Objectives Exchange. Objeoftve-Based Test ColUations. Los Angeles- 
Instructional Objectives Exchange, n.d. $25.00 per test collection plus 3% handling. 

( 



• series of cr i ter ipn-referenced tekts which can be used in diagnosing 

an ndryidual learner's deficiencies, targeting instructional improvement and 

highly sensi ,ve to pupil progress wit^ respect to specific objectives, and-repre- 
sen a useful alternative to the often inadequate standardized t^sts^ The tes? 
collections currently available cover four main areas: . 



(1) 
(2) 
(3) 



Skills^'K-6'^^'"^'"'^" WorOttack Skills, K-6; and cLprehension 



Language Arts, including Mechanics and Usage," K-6; Wo if d Forms 
Syntax, K-6; Composition, Library, and Literary SkilH K-6 
•Social Studies, which includes American Government^ l6-12. 

^#3fQanrl Ml lmk» .-—c 1/ _^ . il . . / 



and 



-Social StudK^s, which mtiuaes American Government,, 10-12 
Mathematics, covering Sets and Numbers, K-6; Mumerati/oh and Relations 
K-b; Measurement, K-6; and Geometry, K-6. ' 



Each test can usually be administered in five to, ten minutes, thereby allowing thfe 
user to combing several tests in one class per led. . 'The< tests are on "pre,4r i nted V 
spirit masters, with betv^een kO and 50 tests pe r col lect io,^; a spi ri t master !s 
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capable of producing 250-300 copies; Each test is keyed to an amplified objective 
-whrchry a detatted description of the learner behavior ^ing measured. A manual 
containing theae atjipl i fied objectives accompan ies, each ' t Jt collection. By con- 
sulting an amplified objecfive, an instructor is/able toUtain a par t i cu I a r I y , 
clear idea of theVkind o6/|earner beh 



V / learner behaviors measured by the test and can thus de- 
sign more relevant >,Wuctional sequences.. All tests are distributed in two com- 
parable versions, one of which can be used as a pretest, 'and theother as a posttest 

V 
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Klein, Stephen Prooedures for Comparing InstruoHonal Programs. Chicaqo' 
American Educational Research Association. 1972. ED 061 27?. 24 paSes 
- Microfiche, i. 65; hardcover. $3.29. pages. 



uativl rrifpH ^°-"parative educational prog ram eva I ua t ion . Sugges'ted eval- 

evl^tfnn f ?"^^^^'"^tion techniques and their weaknesses are discussed. An 
author H?c proposed, and an example of its operation is provided. The " 

oT^han h^r^avf b'eenT instructional programs are more Necessary 

now, than they have been in the past; one reason is that vast numbers of programs are 
now available w th similar content but different object i ves -and va ylng'degr'ee of 

e e%"en;iaTto'a'co!;rrV'^'^'''"- 'actorTth^; 
formanrpon h ^o-^P rehens i ve .program comparison, for example, the student's per- 

IIZZTZ\ prograr^s' objectives and the cost of achieving those objecttves. 

penden tL' "''thL%'H ^T^""^ ''^^^'"^"^ P-^rams. One'is an inde- 

^-pendent Test method that uses a, nationally normed standardized test for all the dif- 

'\?;:s mav^nofbe -^^'T °' '^'^ ^^^^"'^^ ^^at all refe^ant obje - " 

So not nLv?L ^^°"='dered. In addition, nationally normed test instruments genera ly 
ttZJT '"<^°'-'^^tion about student performance on specific objectives. Other 

"sun ThTL '"i^"^ testing, aid Program F r 

lesting. Thp major weaknesses of these techniques is that they fail to take intn ;,r- 
count several important factors that should b^3^cons i dered a they pi ce oo L h 
:rams"'%he" '^^^ --^P-^b i I i ty (or at least partial overlap) of objeclIivL Across pro- 
ef^ec^ivenesr ^-^^ Preser^ts a general formula and procedures for determining program 
abovf The . take into consideration all six of the major factors mentione4 

programs ?s IVJa '^^''^''^ importance of objectives Across 

iec? ve U s nnJ thus-eliminating the necessity for overlapping ob- 

irocldures 1 . H 2 ''"P°^tance, however, that this pa rt i cu lar ^ormul a and^ 

?harmust be ?.k>n' ^'^Sf^t^^ procedures highlight the kinds of factors ' 

that must be taken into cons i de rat ion^f one wishes tc^make val.ixi program comparisons. 
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oni^^o^' and Henrle, Samuel. Educational Aoaountability and'- EvatuaHon • 



Pr^.V^'^ ' V"^ °^ ^ °^ ^ (Puttirilg Research in Educational 

Practice reports. These reports make n^commend^ons but instead inieJpiet 
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the current thinking and practice of researchers and developers in education a: 
objectively as possible. This specific report deals with accountability. The 
four sections of the report treat the philosophy of accountability; accountability 
as a system; specific program approaches to accountability such as external and ? 
internal performance contracting, the voucher system, externa 1 aud i t or EPA 
(External Program AudTt) , and PPBS (Program PI anning and Budget i ng Sys tem) ; and 
last, evaluation as the hea.rt of accountability (i.e., valid as^sessments must be 
made in order to determine whether promises have. been kept). A list of references 
follows three of the four sections. Five case studies ar^ included in this report 
to illustrate specific attempts to implement accountability systems.. The Portland 
Oregon and Grand Rapids, Michigan experiences provide insights into the planning ' 
and development of internal and external contracts applied to elementary 
and secondary education. The Seattle, Washington effort is an example 
of the external program audit tecflnique applied in higher education.^ The 
Syracuse, New York program applied the general systems approach 'to the planning 
and organization of an elementary and secondary educational program in such a way 
that teachers and administrators are accocrfitable for pupil progress. Finally, the 
Hillsborough, California program study describes the use of Program Planning and 
Budgeting Systems (PPBS) to systematically organize a school system so that it 
can more readily be made accountable. 
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Ozenn^ Dan Gilbert. Tmard an Evaluative Methodology for Criterion-Referenced 4^ 
Measures: Test Sensitivity. Los Angeles: Center for the Study of Evaluation, 
(JCLAXGraduate School ^ Education, 1971. ED 061 263. 96 pages. Mkrofiche, $.'65; 
hltts^^ver $3.29. ^ J . 

Th is ftonograp^d iscusses criterion-referenced measures, what they ar^ and how * 
to .evaluateX them."' The author explores the development and evaluation of cri terion- 
referenced \rAeasures and elaborates on the distinction between them and norm- 
referenced njeasures. By elaborating on the bases for distinctions between the two 
types of measures, he points out that methods for evaluating on^ type of measure 
may be inappropriate for evaluating the othero He introduces the concept of sen- 
sitivity as an appropriate method f or . eva 1 ua t i ng the objective-based measure 
(cri terion-referenced'measures) and presents methods' for measuring sensitivity. 
Sensitivity is a measure of how sensitive a test is to the presence of the relevant 
skills. The .traditional model for the response of a subject to a measure is pre- 
sented to show how it leads to* an estimate of the reliability of the norm-referenced 
measure. This model is then extended to conform to the typica) objective-based 
measurement s i tuation to show how the model can be used to evaluate the sensitivity \ 
of a measure.. The author includes table's* summarizing data, from both simulated and ^ 
empirical sources, which ar^ the results, of studying the sensitivity concept under 
a variety of conditions. Finally, the imjil icat ions of the results from the various 
data sources on the sensitivity of the measure are discussed^ This monograph could 
be useful to anyone^ i n teres ted in criterion-referenced and norm-referenced 
measures. A school Evaluation specialist or person functioning in this role would 
find it valuable if fty(^ or her school or district were embarking on a student or 
school assessment progi 
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Arends, Richard I.; Phelps, Jane H.; and Schmuck, Richard A. Organization 
Development: Building Human Systems in Schools. Eugene: The Center for 
Educational Policy and Management, n.d. 49 pages. $1.75. 



Organization development'' 1 s a way of dealing with present and potential 
problems in schools. It is a theory, a strategy, and above al I a process. 
Flexibility and initiative, alertness to the first indications of the need for 
change, and willingness to introduce change are characteristics of a collabora- 
tive organ izat^ion. This booklet reviews the group processes and organizational 
procedures of a particular model which an organization can adopt to become 
collaborative. In this model, consultants come into the school district and train 
a cadre of specialists in organization development. The newly trained specialists 
remain In the district to train others. Training is in collaboration, decisiop 
making,- procedures for meeting, and communication and problem-solving processes. ' 
Brief lectures, reading, practice sessions, and group and intergroup exercises 
make up the instructional sequence. These specialists are drawn from within the 
district and, therefore, are not resented as outsiders. Specialists generally 
provide services in schools ^ther than their own, thus avoiding any conflict that 
might result from being too personally involved in the schooPs problems. Theif" 
consultants do not come in and solve existing prob I ems ; ' i ns tead , they equip local 
people with the skills and procedures for approach i ng .a I I problems. The developers 
recommend training a small" group first; which'in turn wi 11 trai n others until, 
eventuajly,' the whole school staff is involved. The booklet describe^ a sample 
training sequence and sampi e act I v i 1 1 es , emphasizing that these are not hard-and- 
fast arrangements. Training should be adapted to each school situation. Additional 
sources of organization deve'lopment consultation a^e cited and references on the 
subject are listed. A complementary audioslide presentation (73 slides, carousel 
container, and audiotape) is available from the developer for $50. 
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Marks, James R.; Stoops, Emery; and King-Stomps, Joyce. Handbook of Educational 
Supervision: A Guide for the Practitioner^. Boston: Allyn and Bacon, 1971. 
950 pages. $15.00. 



The 'purpose of this book is to provide specific, practica] assistance to on- 
the-job superyisors in the successful realization of their main job: the improve- 
ment of Instructiono The authors emphasize human relations, commun icat ion , and 
teamwork, stressing that supervision is a cooperative service/. The book's 19 
chapters suryey the problems of educational supervision from la variety of viewpoints, 
but always with a f^actical end in mindo Sample chapter titles are "How to Be a 
Successful Supervisor. Through Leadership and Human Dynamics"! ''How to Irfiprove 
Supervlspry Visits"; "How to Improve Supervisory Conferences'^'; and "How to Help 
Staff Understand and Guide Children." Tv4lve of the 19 chapters have supplements 
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that discuss in greater detail some of the issues raised. For example, 
Chapter 13, "How to Measure Teacher Effectiveness and Improve Methods and Tech- 
niques of Instruction," has a supplement that reviews different kinds of instruc- 
tional methods, Also included in the book are simulated "in basket" problems 
which the reader is asked to solve, end of chapter bibliographies, many charts and 
graphs, lists of "do*s and don't's," many suggested techniques and procedures for 
each supervisory task-area, questions for analysis and discussion, and suggested 
class activities. This book would be appropriate for a graduate class in super- 
vision and could be an effective guide for a person in his. or her first supervisory 
job. Experienced supervisors are likely to find much of this book elementary, 
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Schmuck, Richard A., and Runkel , Philip J. Organizational Training for a School 
h'lL^ultih Eugene: The Center for the Advanced Study of Educational Administra- 
tion, University of Oregon, 1970. 191 pages. $2.50. 



This monograph is a case study of an experiment in training a school faculty 
in flexible, organizational problem solving. The premise of the experiment was 
that organizational problem solving could be improved if a faculty were tk-ained in 
group process skills. The experiment emphasizes developing new organizational 
characteristics rather than changing individual brehavnor. Three important assump- 
tions underhy the research design; first, a faculty will adopt new interpersonal 
procedures if they try them out first away from school; second,* greater effec- 
tiveness will result if training deals with actual organizational problems early 
in the program; and third, participation by the whole faculty is necessary for 
optimum benefit from the program. The target audience for thjj typ^of training 
is school personnel at any level, but particularly administrators and teachfers. 
The monograph may be of interest to administrators considering organ i zat ioJa 1 de- 
velopment training in the'rr school. The primary target audience for this monograph 
includes organizational specialists in school districts and trainers of organiza- 
tional special ists." A secondary audience may be professors, researchers, and 
school personnel such as administrators, department heads, unit leaders, teachers 
of all grades, school psychologists, xurriculuni specialists, and counselors; and 
also state department of education personnel. Readers interested in a survey of 
the project should read only the first chapter which summarizes the project and Its 
outcomes. For those wanting more detailed Information, the rest of the, monograph 
describes the preparations made by the developers of the experiment and the exer- 
cises used In training. Sample questionnaires and descriptions of products used 
are also Included. ' f « 
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Schmuck* Richard A.; Runkel , Philip J.; Saturen, Steven L.; MarteU, Ronald T,; 
and 'Derr, C. Brooklyn. Handlfodk of Organizational Development in'SchoolQ. 
Palo Alto: National Press Boiks, 1972. 436 pages. $12.50. 
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As "a guide to planned actions for facilitating human responsiveness and 
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adaptability, in sclnool organizations," tine handbool< is botin a tool for the 
organizational specialist and a reference for others. Chapter Sne descMbes 
technic ^. f of organizational development; Chapter Two spec ie 

and g3rtrcha r"'"?.'"''T"''^"'- ^^^^ constitute a framework 

and gu.de to Chapters Three through Eight. These later chapters study the • 

qoaf^ uLnLv''"''H''°"f.' '^^^^'°P"'«"^= clarifying communication; establishing 
proM^ms °n^:a^ h"°-':^ group meet i^gs ; solving 

usua Tr^lnJn ' decisions. Although the order of the chapters fallows the 

Ever! rlV. ^ sequence, each of the chapters and sections can be used separately 
Every chapter contains the following sections: rationale; insfruments fo? asse 

o ed' rL'Vr" cJu^l''^''' conditions; exercises for simulating the funcUon 
c ses and nZr H ' " ' and suggestions for combining the instruments, ex^r- 

sequencL can bf ^^^^^ \T ''u'"'"^ sequences. Chapter Nine tells how t ese 
eva^uaUna ?n^^rw' together into overall training designs. Information for 

Change ?en ?hir;'T,""- ^^^'^^^'"9 aspects of training designs is in 
fnr I . I I '"tended primarily for organizational specialists and 

and luatln •?^gf"'^ational specialists. Others who^ can use the info mati'on 
eSucation^Ir ^^'"'"'s trators , state and local department of " 

counsi ors LhT! h T'''"'' ° ^d^"''°"^' administration and curricula, school 
izaUoIarrL P=y'^h°'°9'fts, classroom teachers and department heads, and organ- 
za nnn.v'!'!* e^^" """y ^=^d °^ conjunction v^ith Organ- 
■ za ion Development In Schools , by Schmucl< and Miles. This earlier work il^ - 
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WalTen, John Lo Charting the Decision Making Structure of an Organisation 
Portland: Northwest Regional lducational Laboratory, 1970. 14 Ss 

This report suggests charting the decision making structure of an organization 

rLrl 'I'^'^'t''"^ '° P^^'^'^'" °^ 'Charting a line of delegation Th s 

chart shows how any person in an organization influences of participates in that 
organization's decisions. If this chart i s/' cons t rue ted by people who mus coor- 

Ict on ^^h""';'''!^''^ ^"^'^ ""P--d communicatfon ^ d jo sa"s- i 

fac ion. Rather than being handed down from a manager to subordinates dects on- 

The e' developed by the people who will be working togethe 

be c , e'd (2)'d:?:rlr '/rr'-. ^-'^l"? ^he area of decision making to 

the cha t U? H H ^^ decisions; (3) assigning titles to columns in 

r.rnrSr ' ^^^^'^'"9 ^^^^ P^^son participates In the decision; and (5) 
rnH?nn ^^^I^^""!?'" ""^^^ reached during discussions. An outline for 

?s no? T 't"""'\"' °^ decisions is given. The author\mphasizes Jha t 

mos ?to ?e%o vi . " 'rt V develop ?t that contrl te5 

for arouos ni LL'r^ P^ocedi^res summarized In the report appear useful 

?rpL°n?ng°[ns1::cl^::!""' ''''''''' ^^^^'^^'^^ specialists, and otEe%s involve. 
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behavioral objectives, 10, ]7\ 2k, 28, 

38, 73, yA, 79, 8i. 
bibliographies, 76, 77, 78 
budgeting, 56, 77, 86^^ 

charts, 7^, 89 

communication skills, 13, 32, A3, 65, 
. 87, 88, 89 

community involvement, 21, 2k, , S3, 

72 , 79 . 

competency development, 38 
content analysis, 56 
curriculum design, 56, 79 
curriculum selection, 17, 56, 79 

vdata gathering, 28, k3 , ks , 59, 72 
decision making, 13, 28, 32, k3 , 65, 
87, 88, 89 

educational objectives, 10, 17, 2k, 28, 

38, 73, 7k, 8k 
evaluation, .10, 17, 21, 38, 53, 59, 62, 

73, 79, 80, 81, 82., 83, 8k, 85, 86 
evaluation, definition) k, 71 

futures literature, 75, 76, ^7, 78 

games, 13, 17, 56, 62 
goals, 10, 17, 21 , 2^.,. 28, 56, 59, 81 , 
8k 

group dynamics, 13, 32, k3, 65, 87, 88, 
89 

group Instruction, 10, 13, 17, 21, 2k, 
.28, 32, k}, kS, 53, 56, 59, 62, 65 

group process skills, 13, 32, k3, 65, 
87, 88, 89 

group structure, 79, 88, 89 

guides, 56, 8k, 87 

identifying problems, 28, kS , 65 
information gathering, 28, ki , kS , 59, 

72 . 
Instructional design, 38, 56, 79, 81 
interpersonal communications, 13, 32, 

87, 88, 89 

local Issues, 21, 2k, kS , 59, 72, 79 
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'measurement techniques, 59, 73, 80 , 8l , 

82, 83, 8i|, 85, 86 
multimedia instruction, 10, 17, 21, 28, 

32, A3, 53, 62 
mul t iuni t school , 79 

needs assessment, 2A , ^9 , 59, 72, 73 
nonverbal communication , I I , 32 

objectives collections, 73, 7^ 
organization development, 87, 88, 89 

performance objectives, 10, 17, 2^ , 28, 

38, 7A, 8k . 
performance-based training, 38 
problem solving, 13, 28, 32, ^3 , 65, 

87, 88, 89 
program evaluation, 10, 17, 53, 62, 79, 

80, 81, 82, 83, 85, 86, 
program planning, 10, 17, 53, 56, 77, 

79 

program planning, def'rnitloci, 3, 71 
programmed instruct ion, 2A , 56 
purposes, determining, 10, 17, 21, 2^ , 
28, A9, 53, 56, 59, 72, 73, 7^ 79, 

purposing, definition, 3, 71 

resource assessment, 56 

school -communi ty cooperation, 21, 2k, 

^9, 72, 79 
selecting/analyzing tr^aining products, 

k 

setting goals, 17, 21, 2k, 28, 56, 59 
supervision, 87, 89 

technology , 78 

test collections, 8I , 82, 83, 8k, 86 
test validity, 81, 82, 83, 8k, 85, 86 

verbal communication, 13, 32, A3, 65, 
87, 88, 89 

workshops, 10, 17, 21, 28, 32, A3, 53, 
62, 65 
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Alternative Futures for Learning: An 
Annotated Bibliography of Trends, 
Forecasts, and Proposals, 76 

An Annotated Bibliography on Admin- 
istering for Change, 76 

CSE (Center for'the Study of Evalua- 
tion) Elementary School Evaluation 
^ Kit--Needs Assessment, 59 . 
CSE Elementary School Test Evalua-* 

tlons, 82 
CSE Presch oo 1 /Ki nde rga rten Hierai — 

chlcal Objectives Charts, 7^ 
CSE Secondary School Test Evalua-- 

t Ions , 83 
CSE-ECRC (Early Childhood Research 

Center) Preschool/Kindergarten 

Test Evaluations, 81 
Charting the Decision Making Structure 

of an Organization, 89 

Designing Instructional Programs, 56 
Determining Instructional Purposes, 28 
Domain-Referenced Curriculum Evalua- 
tion: A Case Study from the 
MINNEMAST Project, 8I 

Educational Accounfeab 1 1 1 ty and Eval- 
uation: PREP Report No. 35, 85 

Educational Goals and Objectives, 2^ 

Educational Values Assefssment (EVA) 
Questionnaire, 72 

Education for the People, i*9 

Ernstspiel ,13 

Evaluation for Program Improvement, 62 
Evaluation of Comprehensive Planning 
Literature with an Annotated , 
Bibliography, 78 
Evaluatipn^WOrkshop' I --Ao Orienta- 
tion, 53 ) ' 

Handbook of Educational Supervision: 
A Guide for the Practitioner, 67 

Handbook' of Organizational Develop.nent 
in Schools, 88 



I ndividually Guided Education and the 
Multiunit Elementary School: Guide- 
lines for Implementation, 79 
Innovations Evaluation Guide, 8^ 
Instructional Development Institute, 17 
Interpersonal Communications ,32 * 

Measurable Objectives Collections, 73 

National Priorities for Elementary 
Education, 72 

Objectives-Based Test Collections, 8^ 
Observations and Analysis of the 

Literature on Change," 75 : 
Organizational Training for a School 

Faculty, 88 
Organization Development; Building 

Human Systems in Schools, 87 

PPBS (Planning, Programming, Budgeting 
System): For People" Who Don't 
Understand PPBS , 77 

Prentice-Hall Teacher Competency 
Development System, 38 

Procedures for Comparing instructional 
Programs, 85 

Procedures for Nqeds-Assessment Eval- 
uation: A Symposium, 73 

Program Evaluation Packet, 80 - 

Project Leadership, 10 

Research Utilizing Problem Solving, ^3 

Eval- 
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uator Training Program, 21 
Shared Decision Making, 65 
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